{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 keras 做回归任务\n",
    "\n",
    "### 导入数据\n",
    "\n",
    "这里使用了 Boston Housing Price 数据集，一共 506 个样本，其中训练集 404 个样本，测试集 102 个样本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://s3.amazonaws.com/keras-datasets/boston_housing.npz\n",
      "57344/57026 [==============================] - 4s 77us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "((404, 13), (102, 13))"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from keras.datasets import boston_housing\n",
    "\n",
    "(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()\n",
    "\n",
    "train_data.shape, test_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1.23247,   0.     ,   8.14   ,   0.     ,   0.538  ,   6.142  ,\n",
       "        91.7    ,   3.9769 ,   4.     , 307.     ,  21.     , 396.9    ,\n",
       "        18.72   ])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预处理数据\n",
    "\n",
    "对于样本的每个特征的含义暂且不深究，但有一点需要注意，不同的维度取值范围不同，为了更好地训练，需要将其值做归一化处理。做法就是每个维度减去均值，再除以标准差，归一化以后，各个维度的呈均值为 0 方差为 1 的正态分布。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = train_data.mean(axis=0)\n",
    "std = train_data.std(axis=0)\n",
    "\n",
    "train_data = (train_data - mean) / std\n",
    "test_data = (test_data - mean) / std"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2.6016254395785847e-15, 0.9999999999999993)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.mean(), train_data.std()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "\n",
    "def build_model():\n",
    "    model = models.Sequential()\n",
    "    model.add(layers.Dense(64, activation='relu',\n",
    "                           input_shape=(train_data.shape[1],)))\n",
    "    model.add(layers.Dense(64, activation='relu'))\n",
    "    model.add(layers.Dense(1))\n",
    "    \n",
    "    # mse - Mean Squared Error\n",
    "    # mae - Mean Absolute Error\n",
    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用 k 折交叉验证\n",
    "\n",
    "为了评估模型的效果，需要从训练集中划分出一个验证集。当数据量很少的时候，训练数据本就很小，划分出的验证集自然也不能很大。在一个小的验证集上得到的结果就显得不那么可靠。k 折交叉验证将数据分为 K 份，每次使用 K-1 份数据作为训练集，使用余下的 1 份作为验证集。这样训练 K 的模型，验证 K 次，最后求出 K 个评估指标的均值，作为最终评估结构。\n",
    "\n",
    "![width=400](https://wangyu-name.oss-cn-hangzhou.aliyuncs.com/superbed/2019/05/07/5cd0f9253a213b0417e34cf2.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "k = 4\n",
    "num_val_samples = len(train_data) // k\n",
    "num_epochs = 100\n",
    "all_scores = []\n",
    "\n",
    "for i in range(k):\n",
    "    val_data = train_data[i*num_val_samples:(i+1)*num_val_samples]\n",
    "    val_targets = train_targets[i*num_val_samples:(i+1)*num_val_samples]\n",
    "    \n",
    "    partial_train_data = np.concatenate([\n",
    "        train_data[:i*num_val_samples],\n",
    "        train_data[(i+1)*num_val_samples:]\n",
    "    ], axis=0)\n",
    "    \n",
    "    partial_train_targets = np.concatenate([\n",
    "        train_targets[:i*num_val_samples],\n",
    "        train_targets[(i+1)*num_val_samples:]\n",
    "    ], axis=0)\n",
    "    \n",
    "    model = build_model()\n",
    "    model.fit(partial_train_data, partial_train_targets,\n",
    "              epochs=num_epochs, batch_size=1, verbose=0)\n",
    "    \n",
    "    mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
    "    all_scores.append(val_mae)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2.0556031241275297, 2.276286233769785, 2.934723700627242, 2.407703190746874]\n",
      "2.418579062317858\n"
     ]
    }
   ],
   "source": [
    "print(all_scores)\n",
    "print(np.mean(all_scores))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 确定合适的 epoch\n",
    "\n",
    "这里训练了 100 个 epoch，但并不知道多少个 epoch 合适，有可能不到 100 时就发生了过拟合。为了确定多少个 epoch 合适，可以记录下训练过程中在验证集上的 loss，当 loss 上升的时候，就可以认为出现了过拟合，以此就可以确定需要训练多少个 epoch 了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_epochs = 200\n",
    "all_mae_histories = []\n",
    "\n",
    "for i in range(k):\n",
    "    val_data = train_data[i*num_val_samples:(i+1)*num_val_samples]\n",
    "    val_targets = train_targets[i*num_val_samples:(i+1)*num_val_samples]\n",
    "    \n",
    "    partial_train_data = np.concatenate([\n",
    "        train_data[:i*num_val_samples],\n",
    "        train_data[(i+1)*num_val_samples:]\n",
    "    ], axis=0)\n",
    "    \n",
    "    partial_train_targets = np.concatenate([\n",
    "        train_targets[:i*num_val_samples],\n",
    "        train_targets[(i+1)*num_val_samples:]\n",
    "    ], axis=0)\n",
    "    \n",
    "    model = build_model()\n",
    "    model.fit(partial_train_data, partial_train_targets,\n",
    "              validation_data=(val_data, val_targets),\n",
    "              epochs=num_epochs, batch_size=1, verbose=0)\n",
    "    mae_history = model.history.history['val_mean_absolute_error']\n",
    "    all_mae_histories.append(mae_history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "average_mae_history = np.array(all_mae_histories).mean(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 K 折交叉验证过程中，记录下每个 epoch 的 MAE，然后求出这 K 折中各个 epoch 的平均 MAE，而后绘制曲线。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Validation MAE')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4nNWZ8P/vPUUz6l0ukmy5YgO2wRjTOySEmmwgIWUTkpC2aWxJ4U3e1N3sS3r9JUs2JCQhG5JsAoRAgNBNsbHBNu5FbrKsXkdlpJk5vz+eohlpZiQZz8i27s916dJoNJo5ejQ693Pu+5zziDEGpZRSCsAz1Q1QSil1/NCgoJRSyqVBQSmllEuDglJKKZcGBaWUUi4NCkoppVwaFJRSSrk0KCillHJpUFBKKeXyTXUDJquiosLU1dVNdTOUUuqEsmHDhjZjTOV4jzvhgkJdXR3r16+f6mYopdQJRUQOTORxmj5SSinl0qCglFLKpUFBKaWUS4OCUkopV8aDgoh4ReRVEXkoyfduFZFWEdlof9yW6fYopZRKLRuzjz4FbAeKUnz/PmPMx7PQDqWUUuPI6EhBRGqAa4H/zuTrKKWUOjYynT76HvAZIJbmMW8Vkc0i8kcRqU32ABH5kIisF5H1ra2tR9WQnU29fPuxnbSFwkf180opNR1kLCiIyHVAizFmQ5qH/QWoM8YsB/4O3JPsQcaYu4wxq4wxqyorx12Ql9Te1hA/fHIP7aGho/p5pZSaDjI5UrgAuEFE9gO/Ay4Xkd/EP8AY026McU7dfwaclanGeD0CQCSWbtCilFLTW8aCgjHmDmNMjTGmDrgFeNIY8+74x4jIrLgvb8AqSGeEzw4K0ZjJ1EsopdQJL+t7H4nIV4H1xpgHgU+KyA1ABOgAbs3U6zojheGoBgWllEolK0HBGPM08LR9+4tx998B3JGNNvi91qBIRwpKKZXatFnRrDUFpZQa37QJCk5NIaLpI6WUSmnaBAWvFpqVUmpc0yYoODWFiAYFpZRKadoEhZGRgtYUlFIqlWkTFHw6JVUppcY1bYKC1hSUUmp80yYoaE1BKaXGN22CgtYUlFJqfNMmKGhNQSmlxjdtgoLWFJRSanzTJij4tKaglFLjmj5BQWsKSik1rmkTFHTrbKWUGt+0CQp6kR2llBrftAkKI1tna1BQSqlUpk1QEBF8HtGaglJKpTFtggJYowW9noJSSqU2rYKC3+vR9JFSSqUxrYKC1yNaaFZKqTSmVVDweUSv0ayUUmlMq6CgNQWllEpvWgUFrSkopVR60yooaE1BKaXSm1ZBwaopaFBQSqlUplVQsGoKWmhWSqlUplVQ8GlNQSml0ppeQUFrCkoplda0CgperSkopVRa0yoo+LSmoJRSaU2voODVkYJSSqUzvYKCx6M1BaWUSmNaBQWdkqqUUulNq6Cgi9eUUiq96RUUvDolVSml0sl4UBARr4i8KiIPJfleQETuE5E9IrJWROoy2RafRxevKaVUOtkYKXwK2J7iex8AOo0xC4HvAndmsiFaU1BKqfQyGhREpAa4FvjvFA+5EbjHvv1H4AoRkUy1R2sKSimVXqZHCt8DPgOkOj2vBg4BGGMiQDdQnqnGaE1BKaXSy1hQEJHrgBZjzIZ0D0ty35heW0Q+JCLrRWR9a2vrUbfJqzUFpZRKK5MjhQuAG0RkP/A74HIR+c2oxzQAtQAi4gOKgY7RT2SMucsYs8oYs6qysvKoG6TbXCilVHoZCwrGmDuMMTXGmDrgFuBJY8y7Rz3sQeC99u2b7Mdk7FReN8RTSqn0fNl+QRH5KrDeGPMg8HPg1yKyB2uEcEsmX9uvNQWllEorK0HBGPM08LR9+4tx9w8CN2ejDaA1BaWUGs/0WtGsNQWllEprWgUFr0eIGYjpaEEppZKaVkHB77VmwEYzV8tWSqkT2rQKCl6P9etqsVkppZKbVkHB57FGCsNaV1BKqaSmVVDw2kFBRwpKKZXctAoKTk1Bp6UqpVRy0yooaE1BKaXSm1ZBQWsKSimV3rQKClpTUEqp9KZVUPBpTUEppdKaXkFBawpKKZXWtAoKXq0pKKVUWtMqKLjbXOhIQSmlkkoZFETkM3G3bx71va9nslGZ4owUtKaglFLJpRspxF/w5o5R37s6A23JOK0pKKVUeumCgqS4nezrE4LWFJRSKr10QcGkuJ3s6xOC1hSUUiq9dJfjXCEiPVijglz7NvbXwYy3LAO0pqCUUumlDArGGG82G5INbk0hqkFBKaWSmdSUVBHJF5F3ichfM9WgTBoZKWhNQSmlkhk3KIhIjoi8WUR+DxwBrgR+mvGWZYBuna2UUumlTB+JyFXAO4A3Ak8BvwZWG2Pel6W2HXO6IZ5SSqWXrtD8KPAccKExZh+AiHw/K63KEKemMKw1BaWUSipdUDgLawHb30WkHvgdcEIXn73ulFStKSilVDIpawrGmFeNMZ81xiwAvgycCeSIyCMi8qFsNfBY8uuUVKWUSmtCs4+MMc8bYz4OVAPfA87LaKsyRGsKSimVXrpC88oU32oFfpiZ5mSW1hSUUiq9dDWF9cBWrCAAifsdGeDyTDUqU7SmoJRS6aULCv8KvBUYwCoy/9kYE8pKqzLEpzUFpZRKK12h+bvGmAuBjwO1wBMi8nsROSNrrTvGnKCg21wopVRy4xaa7TUKDwCPAauBxZluVKa4W2frSEEppZJKV2iej7VO4UbgEFYK6T+MMYNZatsxJyJ4PaI1BaWUSiFdTWEPsBlrlNADzAH+ScQ62zbGfCfjrcsAn0e0pqCUUimkCwpfZeRiOgVZaEtW+DyiNQWllEoh3fUUvpzFdmSNV0cKSimV0qSupzAZIhIUkXUisklEtorIV5I85lYRaRWRjfbHbZlqj8Pn9ej1FJRSKoV06aPXKwxcbowJiYgfWCMijxhjXhr1uPvsLTSywucR3eZCKaVSyFhQMMYYwFns5rc/prw39nmEiNYUlFIqqXGDgogEsFY218U/3hjz1Qn8rBfYACwEfmyMWZvkYW8VkYuBXcA/G2MOTazpR8fr1ZqCUkqlMpGawgNYaxUiQF/cx7iMMVFjzBlADbBaRE4f9ZC/AHXGmOXA34F7kj2PiHxIRNaLyPrW1tZkD5kwn8ejQUEppVKYSPqoxhhz9et5EWNMl4g8DVwNbIm7vz3uYT8D7kzx83cBdwGsWrXqdfXoPl28ppRSKU1kpPCCiCyb7BOLSKWIlNi3c4ErgR2jHjMr7ssbgO2TfZ3J8mpNQSmlUprISOFC4FYR2Yc1o0iw6sjLx/m5WcA9dl3BA/zeGPOQiHwVWG+MeRD4pIjcgJWa6gBuPcrfY8L8Xg9DUR0pKKVUMhMJCm86mic2xmzGuoTn6Pu/GHf7DuCOo3n+o5Xr9zI4HM3mSyql1AljIrukHgBKgOvtjxL7vhNSMMfLwLCOFJRSKplxg4KIfAq4F6iyP34jIp/IdMMyJdfvYXBIRwpKKZXMRNJHHwDOMcb0AYjIncCLnKDXac71exnQ9JFSSiU1kdlHAsT3olESr9d8QsnN0aCglFKpTGSk8AtgrYj82f76zcDPM9ekzAr6vZo+UkqpFMYNCsaY79gLzy7EGiG8zxjzaqYblim5fi+DEQ0KSimVTLrLcRYZY3pEpAzYb3843yszxnRkvnnHXq7fy3DUMByN4fdmbOdwpZQ6IaUbKfwWuA5rQ7v4JcBifz0/g+3KmNwcLwCDw1ENCkopNUq6K69dZ3+el73mZF7QbwWFgeEohUH/FLdGKaWOLxNZp/DERO47UeTaQWFwSBewKaXUaOlqCkEgD6gQkVJGpqEWAbOz0LaMcNJHOi1VKaXGSldT+DBwO1YA2MBIUOgBfpzhdmVMrl+DglJKpZKupvB94Psi8gljzAm5ejkZt6agaxWUUmqMiaxT+KF9xbRTgWDc/b/KZMMyJX72kVJKqUQTuUbzl4BLsYLCw1hbaa8BTsigEPRbtXVNHyml1FgTmah/E3AF0GSMeR+wAghktFUZlKvpI6WUSmkiQWHAGBMDIiJSBLRwgi5cAy00K6VUOhPZEG+9fa3ln2HNQgoB6zLaqgwKak1BKaVSmkih+Z/smz8Vkb8BRfalNk9Imj5SSqnU0i1eW5nue8aYVzLTpMzyez34PKI7pSqlVBLpRgrftj8HgVXAJqwFbMuBtVhbaZ+Qcv1eBnSbC6WUGiNlodkYc5kx5jLgALDSGLPKGHMWcCawJ1sNzISgXn1NKaWSmsjsoyXGmNecL4wxW4AzMtekzMv1e7XQrJRSSUxk9tF2Eflv4DdY11F4N7A9o63KMCt9pEFBKaVGm0hQeB/wUeBT9tfPAj/JWIuyQNNHSimV3ESmpA4C37U/Tgq5fo8GBaWUSiLdlNTfG2PeJiKvkXg5TgCMMcsz2rIMyvV7ae8bmupmKKXUcSfdSMFJF12XjYZkU26Ol4FOHSkopdRo6a6ncMT+fCB7zcmOoF9rCkoplUy69FEvSdJGWAvYjDGmKGOtyrCgTklVSqmk0o0UCrPZkGzSKalKKZXcRKakAiAiVSReee1gRlqUBbl2+sgYg4iM/wNKKTVNjLuiWURuEJHdwD7gGWA/8EiG25VRuTleYgaGorr/kVJKxZvINhdfA84Fdhlj5mFdhe35jLYqw4J+55oKGhSUUireRILCsDGmHfCIiMcY8xQnwd5HoBfaUUqp0SZSU+gSkQKs7S3uFZEWIJLZZmVWbo4VC7XYrJRSiSYyUrgRGAD+GfgbsBe4frwfEpGgiKwTkU0islVEvpLkMQERuU9E9ojIWhGpm1zzj45ep1kppZJLt07hR8BvjTEvxN19zySeOwxcbowJiYgfWCMijxhjXop7zAeATmPMQhG5BbgTePskXuOo5OVYv3bv4Ak94FFKqWMu3UhhN/BtEdkvIneKyKTqCMYSsr/02x+jF8PdyEig+SNwhWRhjujCqgIAth/pyfRLKaXUCSXdlde+b4w5D7gE6AB+ISLbReSLIrJ4Ik8uIl4R2Qi0AI8bY9aOekg1cMh+vQjQDZQneZ4Pich6EVnf2to6oV8snVnFQSoKAmxq6Hrdz6WUUieTcWsKxpgDxpg7jTFnAu8E3sIEL7JjjIkaY84AaoDVInL6qIckGxUk25H1LvtyoKsqKysn8tJpiQgraorZ3ND9up9LKaVOJhNZvOYXketF5F6sRWu7gLdO5kWMMV3A08DVo77VANTar+MDirFGJRm3rKaYva0hQmGtKyillCNlUBCRq0TkbqyO+0PAw8ACY8zbjTH3j/fEIlIpIiX27VzgSmDHqIc9CLzXvn0T8KQxJtkmfMfcipoSjIEth3W0oJRSjnQjhf8DvAgsNcZcb4y51xjTN4nnngU8JSKbgZexagoPichXReQG+zE/B8pFZA/wL8DnjuJ3OCrLa4oB2Kx1BaWUcqXbJfWy1/PExpjNwJlJ7v9i3O1B4ObX8zpHq7wgQHVJLpu0rqCUUq6JLF47aS2sKuBge/9UN0MppY4b0zooVBQEaA+Fp7oZSil13JjmQSGHtr4hslTbVkqp4960DgrlBTkMRWL06cZ4SikFTPOgUJYfANAUklJK2aZ1UCgvyAGgLTQ0xS1RSqnjw7QOChWjRgp94Qgf+OXL7GubzHIMpZQ6eUzroOCMFNr7rJHC9iM9PLGjhWd2tkxls5RSaspM66BQlm8HBXuk0Ng9CMDBjoEpa5NSSk2laR0Ugn4vhQGfO1Jo7LKCwcEOXdCmlJqepnVQACgryKE9lBgUGjo1KCilpqdpHxTK83No77PTR3EjBV3QppSajjQoFATckcLhLqum0D8UdVNKSik1nUz7oFBRkOOuU2jsGqC6JBeAQ1pXUEpNQ9M+KJTnB+jsH6J3cJjugWHOmV8GaLFZKTU9aVAoyCEaM+xo6gXgnHlWUNCRgjrZ3PfyQe740+apboY6zmlQKLBWNW+2L7azoLKAysIAh3StgjrJPLe7jYdfa5rqZqjjnAYFewHb+v0dAMwuyWVOWZ6mj9RJp3cwQs/gMNGYzqzLtid3NPPi3vapbsaETPugcNrsIsryc3hkSxMegarCADWluTR0aVBQJ5dQOIIx0D0wPNVNmXb+8+EdfPfvu6a6GRMy7YNCSV4O337bCgBmFgXxeT1W8blP/3HUySU0GAGgs1+nW2dbc88ghztPjJS0b6obcDy47JQqvnDtUvrti+2U5PkJhSMMRWLk+KZ93FQniVDYCgpdGhTSGhyOEvR7j+nz9QxG6BuKEonG8HmP7z7l+G5dFt120Xw+ecUiAErz/IAOs9XJpXfQej/rKNiaXfihX62nzw6UjoPt/Zz+pUfZeKjrmL1Wa6+1Y0I0ZmjpTX1Br79uPsK3Ht15zF73aGlQSKI4zyo+nyhnVL2Dw3z1L9sYHNbLiqrkjDHuSOF4Sh99+g+b+NMrDVl/3YdfO8Jj25rZdqQn4f7dLb1EYoYdo+5/PVp6B93bh7tSp5B+89IB7nlx/zF73aOlQSEJZ6TQ2T9yRtXVP3TcztpYW9/B3c/vO6ZnN+rkMjAcxXn7Hi8jYGMMD2xs5MFNjUf18681dPPXzUeO6medKejNPYMJ9zf3WGfyR7oHx/zMZL3/ly/z3cd30dIzMjpoTBEUjDFsaeymdzAy5Sd3GhSSKB01UhgcjnLRnU/x23UHxzy2vjVEd//U/pP1hq3XHxjSkYJKzikyw/EzUgiFIwxFY+w40ntUP/+TZ/bwuT9tPqrNKzc1WCdQzT2J6RznrH50sJislt5BntzRwnO7WxNSRg0pis0H2vvptf9Go/dde3l/B//3/i1Z26RTg0ISxbnWSKHL7uybewbpDUfctQyOWMzw1p+8wPeemNqpZs6bqV+DgkqhJyEopD+JicUM//1cPZtSjDz3tfVx0TeeTJsKSaa7f5hXDna6XzsbUTb1DB7ViVVD5wC9g5Fxf5/ROvqG3M65ZVTn73Tgr3ek8PyeNgDq2/po6R3E6xGKc/0pj9lrh7vd222j6g6/f/kQv37pALtbQq+rTROlQSGJUntBW9eA9aZ1CkXbR+UZD3X209k/POVbYowEhcg4j1TTVSiuoJquVmaM4Wt/3ca//3U733osedHzpfp2DnUMsLNpcnn3Hzy5m5t/+qJ7Nu5sWQ+wY5LPBbhTPCd7TfXNDSPBbvSIwAkSExkpvHKwky/c/xoDQ1F6B4fZ2jjSsT+32woKXf3D7GzqpbLAWv/U2DVA/1BkTIF7S9zPto4KClsarWPz7K7Wifx6r5sGhSTyc7z4POKegThnD3tb+xLyfU6QGD0EzbYee1bJgBaa0/rJ03t55LWjy0Gf6Jz0kUfSzz56aPMRfvH8fioKAqzb15E0v7272Tpjdc70J2rN7jaiMcOjW5vH/PzO5smlkAbitrc/0D7ZoNCNCCyZWZgkfTTxkcKDGxv5zUsH+eCv1nPjj57nhh89T2tvGGMMa3a3uZf7Xbuvg6qiANUluRzuHODWu1/mI7/ZkPBcWw53U2FfM74tNNKmweEou+1j4wSaTNOgkISIUJKX455RxU8pc/4hALbZEbzpdeYfXy9NH03Mz9fU84cN2Z/psqOph7O+9vikO69jKWTXnWYV56atKWw81EXQ7+EbNy0jHInx8qiUKVgzdGBytYnW3rDb8T9sF4edTt0jsKOpl1jMEJvgZI7DcTsO7G+f3Eh9c0MX8yvyWVBZQHPv6JGC9b/ePTA8bo3ucNcAuX4va/a00dA1QDRm2NrYza7mEC29Yd65eg5g/X9WFQaoLs1lT2uIdfs7WL+/0524Yoxhy+EeLl5cCSQGhV3N1myo6pJc1u5rz0oRWoNCCqV5fremED+lLD6F5ExnawuFiURj2W1gHA0K44vGDB19QzQdg1klk/X3bc209w3x6sHUs8P6whGu+PbTbi4arDY7awtS6Rkc5m9bmsYtQjrvkdqyXPd97XhhTxv/9odNGGM42NHPnLI8zp1fTo7XkzRlscfObXdMYr3DC3ut3+uSxZWs3ddOWyhMhx0UltWUsGF/J1d99xm++tC2hN/t6w9vTzpbKr5guz9N+igaMwnp3YbOfp7f087qeWVUFQUSZgZFY4bWUNi9pkqyk73uASsdBFb66rwF5fz2g+fw0CcuBKw+wfkbvv3sWnfxa2VhkOqSXJw/08BwlH1t1nE80N5P98AwZ80tpTDgc6/vArDlsNXHfPCieQwOx3jlwEhNJlM0KKRQkud3z4RaesJUFATI9XsT5jVva+zB6xGMgdbQ1KWQnI5jQGsKKXX2DxEzr39WydFYu886265P03ltO9LD3tY+Xqof2TTtVy/u59JvPp3yhKN/KML7fmGlIpzXAPj8n1/jI79OTE84NYXa0rwxZ/h/fKWBP25ooKU3zMH2fuaU5ZOX4+PseaVjUha9g8NuaqWjL/17fmdTr3s2/MKedoqCPj79xlOIGXhsazNtoTAFAR8raorZ2dzL3tY+7nv5kJtvf3xrM3c9W89dz+4d89xOwXZhVcGYEVgoHHHz+9//+y6u+PYztIestM6XHtiKCHz88kXMKAoSCkfcY9PRZ007X1ZdDMCR7rFF4S/cv4WbfvIC0ZjhsH1RrvMXVLB4RiE1pblsa+zhpfp25pbnUVuWR115HmDtqeYEm6tOnQGMFJeftwPmufPLqSgMJPQlWxq7Kc71c9OqWvxe4dkspJA0KKRgpY+szrY1FGZmcYBTZha6I4XOviEauwc5a24pMLV1BR0pjM8Zkrf3DRGOZO84DUdjbLDP7tKd0Tpnn/G7824/0kN73xDNKVbBfuaPm3n1YCciJASTNXva+NvWpoRRR8gdKeQRjsQS0hBOGnRnU687UgC4aFElO5p6E2bo7G0d+R3SjRQeee0Ib/zes/z4qT0YY3h+bxvnzi/ntNlFVBTksPFQJx19Q5QX5Lid8HvOm8vAcJS/bbG293Y6zV8+v5/OUdM0GzoH8HmE1fPKxqSPvvXoTq7/4RrW7G7jFy/sZygaY/2BTp7b3cYTO1r45ysXU12Sy4wia9v80cXlFbUlCV87WnoHeeS1I/SGI+xs6qV7YJjq0lz3+6fNLmLL4W7W7uvg3HnlAMyvKACgqijA+QsqeOc5c/j6W5YR9HvcUcCa3W3MLg4yvyLfuhJk3N976+FuTq8uoiDg41s3r+DmVTUpj/mxokEhhZLcuPRRT5iqwiBLZxWxrbGHzr4hNzhcdkoVMDVnoI6RkYIGhVTii5otRxHAYzFzVMd3a2MP/UNR/F5JO0smWVBotK8Z3pBkdlskGuOxbc2865y5nDqriLX11khhOBpzUyvfeHSnm1YKhSME/R4q7OuHdMatwXGmOr5U387AcJS59tntefOtjm1dXF3BKXpWl+SmHCk0dPbz2f+1Lubz27UHeWZXKw2dA1yxtAoRYX5lAfWtfbSHhijLz+HNZ1bz6O0X85UbTqO2LJc/v3oYsNYSVJfk0j8c5edr9iW8xuHOAWaVWB1p98CwGzSGozEe3NRIzMD7frmO3sEIXo+wfn8HD21upCjo49YL6gCYURgERk7onNrh8hpnpJD4P33fukNE7JHP07ta3OPgOHVWMfvtVNC5C6yLdc2vzAegqjBIcZ6fr79lGZWFAZbOsgJINGZ4fk8bFy6qQESoKAjQFgqztr6dW+56kdcOd3P6bKs9N55RzYLKgqTH/FjSoJBCaX7OSPqoN0xVYYCbV9UQjsS49Zcvc+ejO/EIXLbEKg5lMyjsaQkl1DmckYIz+ygUnvo0UixmGJ7COsto8cW7o/lb/eipPVz0jafGTBccz1r7DP6qU2ewv60vZe7fCQqHEoKC1bknW/BU39bHUCTGmXNKOGdeOa8c7CQcidLQaRU8V9eVselQFy/vt0YpPYMRCgL+kdX69ll+fIrnie1WR+eMFE6bXURejpd1campPS0hcrwezqgtSbk+4MdP7WE4avjCtUtp6hnk9vs2Mrs4yJvPrAZgQWU+e1tDtIXClOcH8Hs9nDKzEBHhLWfW8PzeNg519LOtsYc3nT6TCxdW8Pi25oTXcFI3deVWp7vfTiE9u6uVjr4h3nPeXIajhosWVbByTgnr9nXw5I5WLjmlCr+9IV1VkRUUnP8l5/Pc8jyKgj4aOge47+WD7jUofrvuIGfYo4ind1q1lviRwqmzi9zb59gjhYVVVic+034tx+mzi9na2MOmhi56BiNcuMjqR6ygMMTPnqtnW2MP71g9h3efOzfpcc4UDQoplOT5CUdi9IUjdPSFqSwMsHJOKT94x5m81tDFoY5+vnfLmSyuKsTrkawGhQ/9ej3f+NvIHPL49NH+tj5WfOWxMQvtsu37T+zmjd99dkrbEC++eDfR2WLP7W7lnhf2A1Zn2BYK84X7X5vUytJ1+zqYX5HP2XVl9IYjCe1wGGPY2dyLR6x29oUjGGNo7E4dFJyUz2mzizlnfhnhSIzNDd1uiuqW1bXASGcZCkcoDPooGbVa35kfX12S684OqrWDgs/r4ay5pQlBYXdLiPmV+VQWBmhPUUfb2dTLitpibj2/jhlFAbr6h/nEFYsI+KydR+dXFNDZP8yB9n53Gqbj5rOs9MhX/rKNcCTGsppiTp1dxL62voTaSkNnPzWledRVJAaFP796mNI8P//3ulP52XtW8c2bVnB2XRmbGrppC4W53D6JA9z0UbObPrJ+n8rCADOLg/xu3UE++7+v8cDGRva2hjjSPcg/njuXmUVBNyVYUzI2KMwtz2O2ff81y2bxnbet4PTqkYABcHp1EaFwhG8/thMRuHBhBWAFhe6BYV6q7+Da5bP5j7csc/8e2ZKxoCAitSLylIhsF5GtIvKpJI+5VES6RWSj/fHFTLVnskpyrTfr3tYQMWMVigCuPn0mj3zqYp7610u5YcVsPB6hqjCQ1ZpCU/egO586GhvZ6GxgKMrBjn6iMcMLk7jK0+/XH6K+dWSq7eGuAf7zke2va6+nBzc1Ut/Wd1yMWiBxpDDRGUh3PVvvXhilLRTGI/Do1mb3jHoidrX0cnp1MfNGdV7xmnvCdA8Ms6rOvj64vShycNjqBBs6x6aPth3pIcfnYX6lFXDAGpU4KSrnPmdkExocpiDgozQ/cV+vrY09FOdGVBVFAAAgAElEQVT6ufQUq7MUgZq4s9+z68rY2dxLd/8wTd2DrNnTxllzSynNy6FnMJJ0NLivrY95FQX4vB7+6dKFrJpbyk1njeTCF1RZx2JgOOrO5XfUluVx6eJK/r7dGhmsqClhYWUBQ9EYh+zgOBSJ0dJrzRKaU5aHzyPsaQkxOBzl8W3NXLd8Nn6vh6tOncHM4qB7LDwClyyucl+rIOAjL8fr/u+29A5Smucn4PMysziXmAGvR9jW2OOmi0+vLmbRjAKiMUOOdyQdBzC7OMjMoiAXLxoJPEG/l39YWYOIJPyeq+eV4/MIz+9p59x55e5xqLT7mVA4wgULy8cc22zI5EghAvyrMWYpcC7wMRE5NcnjnjPGnGF/fDWD7ZkUZ5jtDOsrC0eGf6fMLKTY/j5Yw9BsjRQGh6P0D0XdaXrxnW7/cMRdyOZs+DWe/qEIn/njZv4nbl+n+9Yd5L+eqT/qefX1rSG3c0rWoU2F9lCYGUUBAj7PhP5Wxhi2NfbQ1T9MOBK1zzKr8HkkYauGdCLRGI1dg9SW5bpBYV/r2GPqrOa9aqk1K+Vge3/CxmmpRgqnzCjE7/VQlp/DkpmFrNnTxoH2PgoCPmpKcykM+EaCQjhCQcBHWV7iAqmth7s5bXYRi2cUAlaaI/5aAqvnlWEMrD/QwY+f2kMsZvjIJQsoc4OLvVVF9yD72vro7Buis3+YBXYu/b3n1/HHj57vpmxgpPgKI9dIj/eP51npkqKgj7nleW4KxpkK29g1gDFW6ibH56GuIp9dzSH2tIQIR2KcOz+xM105txQRWDmnNCEIiQgzioI8sb2Zrz+8nWd3tVFl/59/7NIFfOdtK1g1t5QdTT1sP9KL3yvMr8x32zO7JIjHIwnP98DHL+COa5aM+Z1Gm1eRz6YvvYFNX3oDv7ntHPf++JHTefNPsqBgjDlijHnFvt0LbAeqM/V6x5rT6TtFOCeCJzOzKOB2NMPRGI9va2b7kZ6EM+3u/mE++Kv1KXdJTOVQRz8/fWavm7Jw/gl77aAQP4+9fyhKz4AVJOKX8qdTb3dS8XPXN9id3mT3lHE8uWPkTLqh4/i42lRbaIiKAist0DSBUV1Tz6C7uKq1N0xbaIiqoiAVBYEJ1xWOdA8SjRlqS/OoLsnF75Wk01J32WmbK+2pigc7+t0plwsq8zk0KrAaYy2SOi0uh335kipe3t/JxkNd1FXkISJUFgXcPHnvYISCoI/KwgCFQR+7W3oZjsbY0dTLabOLWDTD6ujmjEpVnFFbgt8rfPPRnfzu5YO87exaasvyKMu3C9Z2beIL92/hfb9Yxz77RMIJgsnUlFrHAhiTPgLrbH5ueR5nzS1FRFgwKijU2/P7ncCzeEYBu5t73eniS2cVJjxfca6fT1y+iI9dtnDMa910Vg1RY/jF8/uIxgw3nDEbgHPml/MPK2tYOquInU3Wcy+ssoKwE0Dj6wmOGUVB8nImdu2y/ICP4lw/3rjAUmH3M0tnFSUNmNmQlZqCiNQBZwJrk3z7PBHZJCKPiMhp2WjPRDg7pTr/sFVpgsKMoqA7BP3zK4f54K/W86bvP8c7fvaSO/Vv7b52Ht/WzMOT3Gbhdy8f5P89ssOdu+ws+Ol2g4IVBAqDPgaGou5IoaU3PKEzYueM3nm+SDTGRnuR1ehpgBP1xPYWN197NCOFzr4h/u0Pm9zf5VhoD4UpLwhYf6sk6aOm7kHe9P3n3OOx9fDIepQj3YN09ltBpapoZB75p/+wib+PKoDGczrz2rI8fF4PtWV57uydeBsPdTGzKEhdeR6FAR+HOvo5YgeF1fPKONI9mJBPb+oZpLN/OKGw+YbTZhKNGTY1dLvF18q4ABYKRygM+BARls4sYvuRXnY29dp5+xIWVVkd3eigEPR7ef8F84gZw4LKAj5xudWxOmkoZ/+iLYe72d/e716cPl1Q8Hk9zLXbODp9BFbK5vcfPo9v3WxdJrco6GdGUcANCs5nZ8SxqKqQAx39bDzURa7f6z53vH+5ajGXLakac//HLlvIc5+5nN3/cQ3Pf+7yMYFj6axC+oeivLS3naUzC+3Xs143fubRsVJpB4ILFkzNKAGyEBREpAD4X+B2Y8zoXa9eAeYaY1YAPwTuT/EcHxKR9SKyvrU1O5tCVRUG8AjumzzdSGFGUZDugWEGh6M8s6uVGUUBvnDtUtbt6+Azf7S29nU6mxcnkeuHkX1mnGmU8UHBGOMGhRlFQXukMNKRTiSF5IwUnKCws7mXPnvqZcdRbLHcF47w8v4O3nJmDbl+r5sHHu9nhiIjnd6L9e38cUPDpI9VOtZIIYeZRcGkheZndrWw/UgP6/ZZrxm/QdmOpl6MgcqCHCoLrFWwoXCEP2xo4CsPbU25uMwZJdWWWh3txYsqeWJHC//9XL37mL5whCd3tHDlqdZ0zdqyPA51DtDYPUiOz8PymhKiMZPQZmd++6mzRoLC8upiNxA7QaGqKOjWnkJha6QAsGRWITuO9PCqvQvqmbUlVBTkcN3yWbzhtJljfo87rlnKY/98CX+7/WJmFVsdYXncSKGzb8ht3+/XH8LrkXGLo85ZvvM8o80oCiacKS+sKmCPXffa29JHeX6Ou3Hl4hmFGAN/29LEKTMLE868X6+l9jEeisZYMssJCtZrJAs+r1d1SS4fu2wB7zmv7pg/90RlNCiIiB8rINxrjPnT6O8bY3qMMSH79sOAX0QqkjzuLmPMKmPMqsrKytHfzojyggDfv+VMinP9VJfkpr1mq/MGf6m+nTV72rh4USW3XTSff71qMQ9uamTdvg638123r8NNKx3q6B83P+38IzhpACcoRGKGfnt3RrCCmDNSKAj48HpkQikkZ6m9ExQ2xC2jP5qRwuGuASIxw2mzi6gpzZ3QSOEdP3uJrz+83f3amR8+0d0vYzHD5//8Wsqtno0xtIXCVLrpo8ExM4jW21M3nXUCWxt73LqSM9OnoiBApb3i1DmTP9Qx4F4kJhKNJRzzQ539eARmlVh56s9fu5Rrls3k3/+63Q14j29rZnA4xo1nWJnVOWV5bvqouiTXDShOXcEYw91r9lGWn8Np9vx1AI9HuNKuSTgzcqoKA+4Gbb2DVk0BrI6ubyjKXzY1Up6fQ01pLiLCj9650l1tOx5npNDRF2Z73A6nB9qtxW/+ca5DPN+eb58sfZTMwsoC9raEMMawtzXkppTASh9ZbRlyO/FjZfGMQpwYs2Sm9dzFeX7+8JHzeM95x36qqMcjfPqNS5hTnt0ZRwltyNQTi1Vu/zmw3RjznRSPmWk/DhFZbbfn2J0evk7Xr5jNM5++jL/Y+5qkctmSKsrzc/jKX7bRPTDMRfbGVu88x9oQa1NDF/VtIUSgNxxxO5mvP7yd9969bsyZpjOjaCgS44C9WtMZKcR31N0DwwkjhaFojM6+YSoKclhUVcCmiYwURqWPNhzopKowQI7Pc1QjBSdlVVUYsINC+pFCOBJly+FuNyUQ/xyji7LGGO5/9fCYRWSN3QPcu/agu+gJrOD08zX73MtQhiMxygtyrOMUiY2plzjB0DneWw93c8HCCjwC2+xRQ0VhwJ2K6fxeeTletwD7qxcPcOOPn3frRoc6+plVnOt2kH6vh++87Qxy/V4e2WKlER/YeJjZxUHOmmOtjD91dhF7W0Osre9gVnHQnQnkvN7Tu1p5sb6dT16+kNycxBOVG1bMxiO40x8rCwP0D0VpC1nbNzgjBafjXLevgzNqS8bMjJkIJ73a0TfsXiRniZ1eSZc6crzlzGref8G8hNk76SysKiAUjtDcE7aCQtwirrqKfLdGMbqe8HoF/V43gC2Je+6Vc0opDPpT/dgJLZMjhQuAfwQuj5tyeo2IfEREPmI/5iZgi4hsAn4A3GKydXmhCcrN8SbNe8YL+LzcvKqWfW19CXOOywus/U42N3RT39rHpXawcLYk2NxgXX5vdOf9X8/u5aI7n0xYWNQ8Kn0ETlCwRwp26qCpZ5CiXD9LZhYmTDNNxhjjdrxOUHj1YBdnzS2lLC9n3JHC/ra+MbtaOsGrqihITWneuEHhQHs/MZN8yui+tj4Odw1w6TefYk+LVey7/b6NPLDxcMJzOCOK+H2p7lt3kK89tI2DHf3uauaKggC1die7Je6iJu2hsBscD3X002FvYbKsupiKggA77BlozkghZkZ+/qOXLGBvax8bDnby+LZmjBlJyTV0DlBblph3Dvq9XLiogr9va6ald5Dndrdx/Rmz3Vks77ugjkp7VevsklxmlQTxCGw61MXgcJSv/3U7c8vzeOc5Y89Sz5lfzsYvvcE9o3XqYM77wOnETplRiBMHnC0dJsvv9VAU9NHRF2ZHUw8VBTlcs2wWMLGgsHhGIV+8/tSE2TvpOCOD53a3JsxuctrivOaxHinASGquqjA4/oNPApmcfbTGGCPGmOVxU04fNsb81BjzU/sxPzLGnGaMWWGMOdcY80Km2pNp71w9BxFrpWJ8EFlWXcxL9R209w1x7vxy5lfk82J9Ox19Q+4Mk/g9asDKjXb2D/OrF/e79zlb/MafvXcPDLtX1HKW7Dd1D1opr9JcmuzZL6m0hsL0hiOU5+fQPxQlHIlyuGuA+ZX5lObnpN3bZsOBDi791tPcft/GhHqAk8N2RgpWG636xz/+fC1/GXU93r32CCF+Ro8TFOrbQjy1o4X97f2s3dfhdv67mhODnbNga8eRHjct5OyHs/1IjxtwygsCXLy4kvL8nIRj+4pdWF88o4CDHf1uCmhZdTFVRQHC9u9XYdcUADY1dOMReM95dQR8Hu596YC7zbSTgjrU2e+mf+JdubSKxu5BPvk/r2KAW86e436vMGgtvAKYXZJrnXCcVctv1h7gA/e8zO6WEF++4TR3983RiuLOXp06mLMorcgeKeTmeJln58PPOMqgANbx7OgfZkdTL0tmFrnrASYSFCZr5ZxSSvP8fPdxa93IwqrE7R4W2TOCnNHKsfR/rl3Kvbede8yf93ilK5qPkTnlefzbG07hY5ctSLh/WU2x2ynNryzgwkUVvLi33e1Acnwe1sTtfNgWCrsF4vvtM+K55Xlx6aORjrrH7nBzvB5K7Px3S+8gRUE/s0tyicSMW4uIRGN89S/bEjZlc0YJZ86xOob9bdbCt6rCIGX5/rT75T+6tRmPWIvUPv3HTe79Lb2DFAR85Ad8brGxoWOA3nCE53ZbG7XF22ufxXb0D7lpNKdo2RYacqe37mvtc1M7e0aNgJyz/J7BCI12QHEWiW1rHAkKFQU5BP1e3nXOHJ7Y0eIei/UHOvB7hWuXzaazf5jn97RZAb6m2D07DPg8FAR8bke7qaHL3c/miqVV3L+x0d0X50CHdTGm5p4wNUmCwuVLZtib2HXwztVzxnSi1y2fxTfeupxbzrZWJX/x+lOZV57P83va+eilC9z9tsbjtN2Z8baiZiQAOGfURztSAGstz9bD3exs6mXJzEJWzyvjC9cu5foVs4/6OVMJ+r3csnqO+/cdvQfQ21fV8uGL52ckpVNREBgThE5mGhSOoY9dtpCrT5+VcJ+zAyRYm2NduXQGA8NRfvqMtR3wTWfV8MrBTne74Od2W7Or5lfkMxy1Lq4xtzw/4RKGzj4qTk2hMGitzAQYjhqKcn3udDnnkoXbjvRw9/P7eGjzyJm605meaeez98StySgdJ3305I4Wzl9QwQcvmsdfNjW6dQBnnyggLh/e75797xh1SVNn101jrMBgjDXTZr7dUT690woK+9v73E5876hr1e5r63Nzys5oIT6l5Pxes+2ZM+8+dy4+j/CR32zgvXev42fP1nPW3FK3YPnXzUdYUFlAUdDv/i4VBQFr7r/9dWtvmNl2AfkGuxMszfMztzyPQ3HrDEanj5zje0ZtCfk5Xj55xaIx3xcR3nZ2rbtVQn7Ax13vWcWn33gK/3rV4qR/j2Sctq/d18Hs4qC70R3Au86dwyevWORej/xovP/CeTR2DxCOxFgyqwivR7jtovmv6znTedc5c/AIBP2eMdNBL15cyR3XLM3I6043GhQyzAkKXo9QW2pdvKQw4OPVg13UledxzemziMQMa+2pkM/sbKU8P4ePXGqNOBZUFTCjcGRxXGffMHUV1j93fFDIjVswUxT0jwQFu3NyRh/xi6d2NfcS9Hvc4pxzRa2qwgBl+TkpC82HOvrZ0xLi0lMquWX1HGIGNy3U0jPodpzOWfKhzgG3+LqvLfGSpntbQ+7sjtbeMJ39wwxFYpxnz9OOGWt7gvq2kZHC4a6BhGvc7m/r4/wFVh1nR1MvHX1D9A5GELFGCo9vb2FFbYk7hbGqKMgXrz+NgM/DgfY+PnzJAn74jpXuyKaxe9A9q3aDQlxwcDid9qWnVFEU9HHl0hnUledzsKPfTSGlmpp551uXc8/7V6ed6hxvYVUBH7tsIb5xZvXEK8nz4/da1/s4b0FFQkH5/AUV/MskAkwy1y2fzWO3X8Inr1jE1aePncp6rNWU5nHjGdWcWVs64VqEmjwNChlWak/5m1OWR47PQ47Pw6X2IprTq4tZVVdKwOdhze52YjHDs7vbuHhxJVcsqcLrEU6ZUcCMoqA7g6Sjf4i5ZfmIWOmj3sFhCoN+d6QAUJTrdzssZ/vl1+ygEJ8+2tzQzWmzi91N0naPGil0DwwnnYP/lH32fvmSKhZUFrC8pthNdbX0ht3dJ0vz/BQGfBxs73NHCjEzsiDQGMPelpC7CKstNHJltNXzytxgcfHiSg6291PfFnLPQutb+2jsGiAciXKoc4Bl1cXUluWy7UiPW09YXVdGY/cgmw518YZRUy3/8dy5PPDxC3n605fx2auXUFkYSJgGeEatFcwr7d+l0p46mR/wkW8fa+cYB/1eHvrERfzf609lbnkeB9v7efVgFx6xtkRJZvGMQnevo0wREbcGcl6GFkPNKc/jX65a7E53zbRv3bwiYVsIdexpUMiCD1w4j3edM1JMdOaCL6suJuj3cnZdGWv2tLLhoHXhkcuWVFFeEODe287hw5csYEZRgGjM0B4K02lfmKQg4EscKcStoygKWjn9kjy/ey3bzfZsGSetEonG2NrYzfKaYrej3RsXFMryczCGpJdCfGpHC3Xlee5UvTefUc2Wwz3sbu61rz1hdUQiwtyKPPa39yfsTe9MYWzuCdM3FHW3GW7rDdPU46RdrCtXFQZ8XH3aTCIxQ1toiMvsjdv+tvUIF3/jKe7402tEY4Z5FfksmVnEjiM9buC7dvlIKs+Zw59OUXBka+nlo0cKcSME5+x+dvHIbJQ55XkUBf3MKcujZzDCI68dYVlNSULhdyo4QS1TQSHbvB45povT1FgaFLLgfRfM47aL5rtfX7m0ilvOruU6Oxd94aIKdjWHuOeF/eT4PFxujyTOnV9uT4O0/rH3tIaIxAxl+TkU5/rpHrB2rqwsDIwZKYCVQ2/sGmRwOMqu5l4Kgz46+60VqLtbQgwOx1hRU0JJ3Nm3tXOkz021jC42hyNRXqrvcC8yDiOd7182NTIwHHVX1gLMLc/ngD1ScC5pur2ph57BYR7fZhWdz5ln7+gZCtPUbRWFZxYFuWrpDN6ysjqhyHfJKZV4PcJ/PVNPJGb40yvWCKWuIp9l1cXUt/Xx/J42PAJvONVKacwpy3PrBeOZU5ZHjtfjzklPFxRmJdnmwEkX7W4Jcf5x0BHPKbM2lMvElgzq5JSdMZ9KkJfj4/+9dbn7tbOu4aHNR7hyadWYobjTyTpn2E5QONQ5wOGuAd49c27CJlzO2ensklzrYiX25nxvOn0mv1/fwL72PncPnuU1xW4QGYrG3E2+yuIWJ8V79WAXA8NRt81W+4LMr8x3V/bGz+euK8/j0S1NVJf2U10SRER4ZlcrD2xspKNviBx7z/6g30Nbb5j+oSgiVsf7BXtqZvy+/QsrC5lbnkd9ax9XLq3iyR0txIxVmJ9VHOQHT+zmT68eZk5ZHjOLg6yoKebKpTMmvEDrgoUV1JTmuXv/OymimXGjAicoJOto44u5x0NQ+MoNp2X18qPqxKcjhePAqbOK3LUNo2cvgdXpwsgWy6V2UHDm0y+ZVZiwurUo1woQNaW5NHYNuPUEZyuFfa19bGropjDoo648H7/X4+bJnRz0yDYGiSOFNbvb8HqEc0d1eOfMK3dz+fGbB84tzycSM2w82MXMYuuSpvWtfXg9wi9uPZsX7riccntRWFsoTFP3AJUFgYRtEsryc9w59nPK81hUVYDfK3ztzafz7nPnUluWS2l+DrNLcrnOHrU4nfMDH7+QTySZ4ZPKZ65ewo/ftTLh2N972zm8deXI9QCcYzQ72UjBLq77vcKquZmtGUxEWX6Ou1+RUhOhQeE44PEI5y+wLrpx5dKxc9Cd1MWmQ1bnXpZnBYXhqDUv/tRZRYnpI3ekEKQ3HOH+jYepLAxwdl0ZXo+wv72PzQ1dLK8pdmdxOMXmSntUUpYiffTcnjZW1BSPyZWfO3+kA6yKSx85m7P1DUWZVZzLxYsqmF0c5NcfWM1lS6rc362iwNpTqKknnHBWDlZtYl5FvjtC+uerFvP/vessZhXn8uXrT+Ox2y9xH+uk6Y7lAqoLFlYkBN3zF1Zw0aIKt/4QLz/go6IghzNrS8dsQ6HUiUDTR8eJz7xxCW89q8btnOPl+DzMLc9zV6ZWFAbc4nBpnjWXXkTwe8VepzCSPgIr5fPF604lx+ehtjSXZ3a1suNILx+8eKTOUZTr53DXwMhIwW7Hn15p4O41+7j71rMpCvp5raGLj18+9sw7/sImlaPSR45ZxUHetGwWV58+c0w6p7IgwN7WEN0DkaRF0euWz3YvT7lkZpG7lYPHIwmd7+nVxXzzpuUZndnzxtNm8sYku4k6vnbj6e4MLKVONBoUjhNzyvPS7oz4wMcu4NWD1v431SW5blBYMrPI7WCDfi/D0Yj7PSfnfUZtCe89vw6wCrJP72ylqjCQcEHwYjvl5OTLg34veTle98LvP3uunrryfGIGLlk8dqfaGfb1AI50D7qpHuf5cv1eBoaj7gggWX6/ojDAY/a1Ca5fPjaFFh/AxnPzqtoJPzYT3rRsbPuVOlFoUDhBlOTlJFwkxBkNxG8AlpfjJTwcI2Dvi3Pq7CLesbqWD140353Gt6y6mFcPdvGrD6xOKJQ6gSS+HlBVaO37s6y6mN+vP0TA5+XChRWsnJN8a4SrT5/FhgMdCZ2+iDC3PI8dTb1pc9sjaaScpBdDUUplhwaFE5Q7Uojbzjcvx0c0ZtxOOeDz8p//sDzh526/cjEfvmTBmBlOJbmJFw4H+OE7VlKS5yccifLYtmaGIjG+cN3SlDN5Pnv1KUnvryvPt4NC6pSK87r/sLJm3L34lVKZo0HhBOXMSIrfWynX72W8iZdejyRdfepckzp+OumympHnvu3CeRTn+t1cfjKpgsWCqnz8OyShAD3a0pmFBP0e3n721KZ+lJruNCicoC5fUsWf/un8MekjZ2O4yXJGHqn24nHWDByN2y6cz6WnVLlz/5NZVVfGa19+o44SlJpiGhROUF6PsNLe3dTxoYvnk+byCWm9+cxq8nO8E96gbTJK83M4O3/82UAaEJSaehoUTiLJLro+UdUludx6wbxj2Bql1IlIT82UUkq5NCgopZRyaVBQSinl0qCglFLKpUFBKaWUS4OCUkoplwYFpZRSLg0KSimlXGLMUS6BnSIi0gocOIofrQDajnFzjgVt1+Qdr23Tdk3O8douOH7b9nraNdcYM3bf+1FOuKBwtERkvTFm1VS3YzRt1+Qdr23Tdk3O8douOH7blo12afpIKaWUS4OCUkop13QKCndNdQNS0HZN3vHaNm3X5Byv7YLjt20Zb9e0qSkopZQa33QaKSillBrHSR8URORqEdkpIntE5HNT3JZaEXlKRLaLyFYR+ZR9/5dF5LCIbLQ/rpmCtu0Xkdfs119v31cmIo+LyG77c+l4z3OM23RK3DHZKCI9InL7VB0vEblbRFpEZEvcfUmPkVh+YL/vNovIyiy365sissN+7T+LSIl9f52IDMQdu59muV0p/3Yicod9vHaKyBuz3K774tq0X0Q22vdn83il6h+y+x4zxpy0H4AX2AvMB3KATcCpU9ieWcBK+3YhsAs4Ffgy8G9TfKz2AxWj7vsG8Dn79ueAO6f4b9kEzJ2q4wVcDKwEtox3jIBrgEcAAc4F1ma5XW8AfPbtO+PaVRf/uCk4Xkn/dvb/wSYgAMyz/2+92WrXqO9/G/jiFByvVP1DVt9jJ/tIYTWwxxhTb4wZAn4H3DhVjTHGHDHGvGLf7gW2A9VT1Z4JuBG4x759D/DmKWzLFcBeY8zRLFw8JowxzwIdo+5OdYxuBH5lLC8BJSIyK1vtMsY8ZoyJ2F++BNRk4rUn2640bgR+Z4wJG2P2AXuw/n+z2i4REeBtwP9k4rXTSdM/ZPU9drIHhWrgUNzXDRwnnbCI1AFnAmvtuz5uDwHvznaaxmaAx0Rkg4h8yL5vhjHmCFhvWKBqCtrluIXEf9SpPl6OVMfoeHrvvR/rjNIxT0ReFZFnROSiKWhPsr/d8XK8LgKajTG74+7L+vEa1T9k9T12sgcFSXLflE+3EpEC4H+B240xPcBPgAXAGcARrOFrtl1gjFkJvAn4mIhcPAVtSEpEcoAbgD/Ydx0Px2s8x8V7T0Q+D0SAe+27jgBzjDFnAv8C/FZEirLYpFR/u+PieAHvIPHkI+vHK0n/kPKhSe573cfsZA8KDUBt3Nc1QOMUtQUAEfFj/cHvNcb8CcAY02yMiRpjYsDPyNCwOR1jTKP9uQX4s92GZmc4an9uyXa7bG8CXjHGNNttnPLjFSfVMZry956IvBe4DniXsZPQdnqm3b69ASt3vzhbbUrztzsejpcP+AfgPue+bB+vZP0DWX6PnexB4WVgkYjMs882bwEenKrG2PnKnwPbjTHfibs/Pg/4FmDL6J/NcLvyRfDcECQAAAMKSURBVKTQuY1VpNyCdazeaz/svcAD2WxXnISzt6k+XqOkOkYPAu+xZ4icC3Q7KYBsEJGrgc8CNxhj+uPurxQRr317PrAIqM9iu1L97R4EbhGRgIjMs9u1Llvtsl0J7DDGNDh3ZPN4peofyPZ7LBtV9an8wKrQ78KK8J+f4rZciDW82wxstD+uAX4NvGbf/yAwK8vtmo8182MTsNU5TkA58ASw2/5cNgXHLA9oB4rj7puS44UVmI4Aw1hnaR9IdYywhvY/tt93rwGrstyuPVj5Zud99lP7sW+1/8abgFeA67PcrpR/O+Dz9vHaCbwpm+2y7/8l8JFRj83m8UrVP2T1PaYrmpVSSrlO9vSRUkqpSdCgoJRSyqVBQSmllEuDglJKKZcGBaWUUi4NCkrZRCQqibuyHrNdde3dNqdyPYVSE+Kb6gYodRwZMMacMdWNUGoq6UhBqXHY++vfKSLr7I+F9v1zReQJe3O3J0Rkjn3/DLGuYbDJ/jjffiqviPzM3iv/MRHJtR//SRHZZj/P76bo11QK0KCgVLzcUemjt8d9r8cYsxr4EfA9+74fYW1dvBxrw7kf2Pf/AHjGGLMCa9/+rfb9i4AfG2NOA7qwVsuCtUf+mfbzfCRTv5xSE6ErmpWyiUjIGFOQ5P79wOXGmHp7w7ImY0y5iLRhbdMwbN9/xBhTISKtQI0xJhz3HHXA48aYRfbXnwX8xph/F5G/ASHgfuB+Y0wow7+qUinpSEGpiTEpbqd6TDLhuNtRRmp612LtYXMWsMHerVOpKaFBQamJeXvc5xft2y9g7bwL8C5gjX37CeCjACLiTbf/voh4gFpjzFPAZ4ASYMxoRals0TMSpUbkin3BdtvfjDHOtNSAiKzFOpF6h33fJ4G7ReTTQCvwPvv+TwF3icgHsEYEH8XalTMZL/AbESnG2vXyu8aYrmP2Gyk1SVpTUGocdk1hlTGmbarbolSmafpIKaWUS0cKSimlXDpSUEop5dKgoJRSyqVBQSmllEuDglJKKZcGBaWUUi4NCkoppVz/P3xDEL4Iul6NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到大约在前 10 个 epoch 过程中 loss 显著下降，但之后就看不太清楚了。一方面是因为前面的 loss 太大，后面的 loss 不在一个量级，因此后面的 loss 在纵轴上体现不出差异来。另外是毛刺太多。为了更容易观察，可以将曲线做平滑处理，并且抛掉前 10 个 epoch 的数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Validation MAE')"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd85XWV+P/XSe+9TXqmM71khjoiIFUQEFQsiKiLqLjouvafZdfyXddd26KLqNgWBaQrCIxIH5jeeyaT3nuv9/z++HySyYSUm0xubpI5z8fjPnLzvu9778lNcs99d1FVjDHGmPEE+DsAY4wxs4MlDGOMMV6xhGGMMcYrljCMMcZ4xRKGMcYYr1jCMMYY4xVLGMYYY7xiCcMYY4xXLGEYY4zxSpC/A5hKSUlJmpub6+8wjDFm1ti5c2edqiZ7U3dOJYzc3Fx27Njh7zCMMWbWEJFib+tal5QxxhivWMIwxhjjFUsYxhhjvGIJwxhjjFcsYRhjjPGKJQxjjDFesYRhjDHGK2d9wuju6+fel0/w6vFaf4dijDEz2lmfMEICA/jlK4U8sbvC36EYY8yMdtYnDBEhPzee7UUN/g7FGGNmtLM+YQBsyE2gpKGDquYuf4dijDEzliUMYGNeAgDbrJVhjDGj8lnCEJEsEXlRRA6LyEERuXuUem8XkT1unZeHlBeJyH73Np/uKLhsXgyRIYFsP2kJwxhjRuPL3Wr7gM+r6i4RiQZ2ishmVT00UEFE4oCfA1epaomIpAx7jEtUtc6HMQIQFBjAuhwbxzDGmLH4rIWhqpWqusu93gocBjKGVfsA8Jiqlrj1anwVz3g25iZwpKqVpo4ef4VgjDEz2rSMYYhILrAW2DrspsVAvIi8JCI7ReTDQ25T4Hm3/A5fx7g+Nx6A3aVNvn4qY4yZlXx+gJKIRAGPAp9V1ZYRnn89cBkQDrwhIm+q6jHgQlWtcLupNovIEVV9ZYTHvwO4AyA7O3vSca7KjCNAYHdJE5csGd4zZowxxqctDBEJxkkWD6jqYyNUKQOeVdV2d6ziFWA1gKpWuF9rgMeBjSM9h6rep6r5qpqfnOzVKYMjigoNYnFqNHushWGMMSPy5SwpAX4NHFbVH45S7Ulgk4gEiUgEcC5wWEQi3YFyRCQSuAI44KtYB6zNjmNPSSMej/r6qYwxZtbxZQvjQuBW4FJ3auweEblGRO4UkTsBVPUw8CywD9gG/EpVDwCpwGsistctf1pVn/VhrACszYqnpauPk/Xtvn4qY4yZdXw2hqGqrwHiRb0fAD8YVlaI2zU1ndZmxwHOOMaC5KjpfnpjjJnRbKX3EAuSo4gODWJPaaO/QzHGmBnHEsYQAQHCmuw4tp+0hGGMMcNZwhjmwoVJHK1upabFNiI0xpihLGEMs2lREgCvHvf5jiTGGDOrWMIY5py0GJKiQuwEPmOMGcYSxjABAcJFC5N4raCOh7eX8u2/Hhr/TsYYcxawhDGCTYuSqWvr4YuP7uPXr52ksd02JDTGGEsYI7h4STLpsWGcPz8RwBbyGWMMljBGlBQVypavXMZ3b1wBwMlaSxjGGGMJYwxZCREEBggn6yxhGGOMJYwxBAcGkJ0QYQnDGGOwhDGuvKRITtS2+TsMY4zxO0sY48hLiqSovt22PDfGnPUsYYwjLymSrl4PVbZViDHmLGcJYxzzkyIBbBzDGHPWs4QxjrxkJ2EUWsIwxpzlLGGMIy0mjPDgQApt4NsYc5azhDEOESE7IYLShk5/h2KMMX5lCcMLmfHhlDV2+DsMY4zxK0sYXshKiKCssRNVm1prjDl7WcLwQmZ8OG3dfTR39vo7FGOM8RtLGF7IjA8HoKzRxjGMMWcvSxheyIyPALBxDGPMWc0Shhey3IRhM6WMMWczSxheiAkPIjo0yFoYxpizmiUML4gIme5MKWOMOVtZwvBSZnw4pdbCMMacxXyWMEQkS0ReFJHDInJQRO4epd7bRWSPW+flIeVXichRESkQkS/7Kk5vOYv3bC2GMebsFeTDx+4DPq+qu0QkGtgpIptV9dBABRGJA34OXKWqJSKS4pYHAj8DLgfKgO0i8tTQ+063rPgIOnr6aezoJSEyxF9hGGOM3/ishaGqlaq6y73eChwGMoZV+wDwmKqWuPVq3PKNQIGqFqpqD/AgcL2vYvVGdoIzU+pknW1CaIw5O03LGIaI5AJrga3DbloMxIvISyKyU0Q+7JZnAKVD6pXx1mQzrVZlxQKwq7jJn2EYY4zf+DxhiEgU8CjwWVVtGXZzELAeeCdwJfB1EVkMyAgPNeLggYjcISI7RGRHbW3tFEZ+upToMHISI9he1OCz5zDGmOF2Fjfw9L5Kf4cB+DhhiEgwTrJ4QFUfG6FKGfCsqrarah3wCrDaLc8aUi8TqBjpOVT1PlXNV9X85OTkqf0BhsnPSWBncaMNfBtjplR9Wzc3/+8Wtpyoe8tt//XcMb786D76Pf5/3/HlLCkBfg0cVtUfjlLtSWCTiASJSARwLs5Yx3ZgkYjkiUgIcAvwlK9i9daG3Hjq23vsuFZjzJRRVb72+AF2FDfyi5cLT7ut36PsLWuitbuP4zWtforwFF/OkroQuBXYLyJ73LKvAtkAqnqvqh4WkWeBfYAH+JWqHgAQkbuA54BA4H5VPejDWL2Sn5sAwI6iRuYnR/k5GmPMbFZQ08on/rCT1JgwtpyoJzcxgleO11LR1El6nLPh6bHqVjp6+gHYWdzI0rQYf4bs01lSr6mqqOoqVV3jXp5xE8W9Q+r9QFWXqeoKVf3xkPJnVHWxqi5Q1e/6Ks6JWJAcSXxEsI1jGGPO2JYT9ZyobedkXTubFiXx29s3ogqP7CwbrLOrpBGA0KAAdhY3+ivUQb5sYcw5IsJ6dxzDGGPOxMm6dsKDA9ny5UtxevDhwoWJPLS9lE9fspDAAGF3SRMJkSHk58Sza9j7TmtXL19/4gCVzV2EBQfyu49u9HnMtjXIBG3Ijaewrp26tm5/h2KMmcWK6trJTYocTBYA79+YTXlTJy8ecZak7S5pZG1WHOtz4imq7zjtfefZA1U8saeCnn4PseHB0xKzJYwJGjqOYYwxk1VU30FeUsRpZVcuT2NebBj3v36S5o5eTtS2szbbSRhw+vvO84eqSY8N47FPXsBP3792WmK2hDFBKzJiCAkKYIeNYxhjJqm330NJQwd5SZGnlQcHBnDr+TlsOVHPJ/5vBwDnzk9kRUYsCZEhfPeZQ1Q2d9LZ08+rx2u5YnnaaS0UX7OEMUGhQYGsyYxjh41jGGMmqayxk36PkpsY+Zbb3r8hm7DgAHYUNfKNa5eRnxNPWHAgv/nIBhrbe/nAL7fyhzeL6Or1cMWy1GmN2xLGJOTnxnOgvJlOd7qbMcZMRJG7lmt4CwMgPjKE396+kSfvupCPXpQ32IJYnRXHb2/fQGNHD9975gix4cFsyEuY1rgtYUxCfm48fR5lT6ntK2WMmbiTYyQMgPPmJ7I8PfYt5fm5CfzlrovYmJfAB8/NJjhwet/CbVrtJKzPdrL6zuIGzl+Q6OdojDGzxdGqVl4rqONkXRvRYUGTOiohKyGChz9xvg+iG58ljEmIjQhmSWo0222mlDFmAu57pZBHd5UREhjA0nnR0zpgPRWsS2qS8nOdhTQzYUMwY8zssLO4gciQQHr6PSMOeM901sKYpPzceB7YWsLRqlaWpft3fxdjzMxX19ZNUX0HX7l6KcC0D1hPBUsYk5Sfc2ocwxKGMWY8A1sK5efGsz5n9iULsC6pScuMDyctJszGMYwxXtlV3EhIYMCIs59mC0sYkyQi5OfG24pvY4xXdhY3siIjhrDgQH+HMmmjJgwR+eKQ6+8Zdtv3fBnUbLEmK46K5i4a2nv8HYoxZoZRVZo6nPeG7r5+9pU3D+5FN1uN1cK4Zcj1rwy77SofxDLrZCU4G4eVNXb4ORJjzEzzl32V5H/n72wtrOeh7aX09Hm4aGGSv8M6I2MlDBnl+kjfn5Wy4gcSRqefIzHGnAlVpaCmbcTb/vBGEc8frJrwY/55Ryl9HuULj+zjh5uPccGCRDYtmrsJQ0e5PtL3Z6WMeOcYxdIGa2EYM5u9cryOd/zw5cFzKAaoKv/53FE+99AeKpq8/2BY29rN6wV1bFqURElDBy2dvXzjumWzbqHecGNNq10tIi04rYlw9zru92E+j2wWiA0PJiYsyFoYxsxyB8qbAfiffxzn7UuSB9/Ya1u7ae3qA+DrTxzgV7fle/Wm//S+CjwK37h2GW8U1iPg9/O4p8KoCUNVZ+9Q/jTKSoig1MYwjJnVjle3ArCrpIk3C0/tETfQTXXJkmReOFLDp/+4i29et5zUmLE/Mz+1t4KladEsSnUuc8WEptWKSKSIfFBEnvZVQLNNZny4tTCMmeWOVbdx3vwEkqJC+flLBYPlx92E8d0bV/KvVyzm74dr+Mhvto/5WAU1rewqaeLGtRk+jdkfxk0YIhIiIjeIyMNAJfAO4F6fRzZLZMVHUNbYgaoN6xgzFUobOvjw/dv4f88cHvzkD+DxKDWtXfT1e6b0+fo9yonaNlZmxPLxTXm8eryOve7RBQU1bUSHBjEvNoy7Ll3EZ9+xiMOVLTSOMZX+j1tLCQ4UblqfOaVxzgRjrcO4XETuB04CNwN/ABpU9XZV/ct0BTjTZcaH09Xroa7N1mIYMxV+tPkYWwrquP/1k7zvvjfp6Onjmf2VLPvms2z87gt87fEDU/p8pQ0ddPd5WJQSzQfPzSYmLGiwlXG8ppWFqVGD4xZrs5yztfeWjXwWTldvP4/sLOXK5WkkRYVOaZwzwVgtjOeABcBFqvohN0lMbWqfA2wthjFTp6iunSf2lHP7hbn86Z/Oo6G9h9+8XsS//eUguYmRrMqMZUth3ZQ+50C306LUKKLDgvnIhXk8d7CaY9WtFNS0szA5arDuysxYRGBvafOIj/X0vkpauvr4wLnZUxrjTDFWwlgPvAn8XUQ2i8jHABsIHybTXYtRauMYxpyxn71YQHBgAP/0tvnk5yawMS+B/3r+KNUt3Xz7hhW8c+U8Shs6qW/rnrLnPOZ2ey1McRLD7RfkEhUaxNefOEBdW/dgOUBUaBCLUqJOa2HsKmnkaJXzGM/sryQzPpzz58/Ng9VGTRiqultVv6SqC4BvAWuBEBH5m4jcMV0BznSZ7loMa2EYc2ae3lfJn3eWcet5OaREO7OQPn3JQlThHeeksiE3gTVZccDoXUKTUVDTxrzYMKLDggHnTO27Ll3I1pPOPnGLUqNOq786M469pU2oKg3tPdz262187qE99PR5eKOw/rRpuXONV7OkVPV1Vb0LyAB+DPjnfMAZKDLUOWZx4FB3Y8zEHalq4V//vJd12XF84aolg+VvW5TE925cyXduWAHAioxYAgT2lExdwjhW3fqWqa+3X5hLTqLTe7Aw+fTbVmfFUd/eQ1ljJ/f8o4DW7j4OVbbw2K4yOnr62bQoecpim2nGGvReN/wCrAFqgf8Z74FFJEtEXhSRwyJyUETuHqHO20WkWUT2uJdvDLmtSET2u+U7JvnzTYsLFiTyzP6qwY3GjDET8/D2MhTl3lvXExp0qudbRPjAudmkxTotjsjQIBanRrOnbOQxhImqbuniSFUrqzNP33I8NCiQ79+0ipvWZQ72IgwYaOX81/NH+cObRVy6NAWA/3j2CIEBMriGYy4aa6X3DuAgToKA0/ePUuDScR67D/i8qu4SkWhgp4hsVtVDw+q9qqrXjvIYl6jq1I5w+cBdly7kr/squf+1k/zLFUvGv4Mx5jTHa1pZmBI12BU1lrXZcTyzvwpVPeOun0d2ltHvUd697q1TYM+bn8h5I4xFLEmLJishnCf3VBAXEcx3b1zBZ/64mx3FjeTnxBPjdm3NRWMljM8DNwGdwIPA46o68u5cI1DVSpx1G6hqq4gcxunSGp4wZr2laTFcvSKN37xexMc2zSc2fO7+wRjjCwU1bSO+OY9kdWYcf9pWSlF9B3lJkz8XW1V5eEcp5+YlTOhxggMDeOULl+BR51N0QIDwzlXz2FHcOKe7o2DsQe8fqepFwF1AFvCCiDwsImsm+iQikoszaL51hJvPF5G97mD68qEhAM+LyM7ZMMj+8U3zae3u49XjteNXNmYOaGzvoa2774wfp627j8rmrtNmI41lpdt9dLDiVLdUX7+HI1UtdPX2j3lfj0cHF9m+WdhAcX0Ht2zMmnDMIkJggBAQ4LRwrl+TwcWLk7lhbfqEH2s2GfdMb1U9KSJPAuHArcBiYI+3TyAiUcCjwGdVtWXYzbuAHFVtE5FrgCeARe5tF6pqhYikAJtF5IiqvjLC498B3AGQne2/uc8rM2IJCQxgf1kz166a2380xgB86NdbWZQSxY9vWTup+/9uSxG9/R42uIcKeZswFiRHESBwvPpUh8fju8v5wiP7CAoQPnXJQv7l8sUj3vefH9xNXVs3939kA//53BHiI4K5esW8ScU/VEJkCL/76MYzfpyZbqxB7/ki8lUR2Qr8G7AXWKqqD3v74CISjJMsHlDVx4bfrqotA91cqvoMECwiSe73Fe7XGuBxYMTfhqrep6r5qpqfnOy/5mBIUADnpMdM6XQ/Y2aq1q5eDla0TPpM+8rmTr779GF+tPkYhyqdz5HeJoyw4ECyEyI4XnNq25B9Zc1EhQZxzrwYntxTPuL9jlS18Nd9lbxZ2MDVP3mV3SVNfOtdy2f1kanTbaxptQXAe4FngTeAbOBTIvIvIvIv4z2wOKNRvwYOq+oPR6mT5tZDRDa68dS7mxxGu+WRwBXA1O4H4AOrMmI5UN6Cx2P7Spm5Y6R90va724GXN3VOanbgz14soKffQ3tPP79/o5jgQCHH3TXBG4tSo09rYRytamVpWjTvXDWP4vqOEfd6+tWrJwkPDuT2C3Mpru/gmpVpvGu19QZMxFgJ499xPtl7gCggethlPBfidGFdOmTa7DUicqeI3OnWuRk4ICJ7gZ8Ct6jz15kKvOaWbwOeVtVnJ/HzTatVmbG0dfdRaGsyzByx7WQDF/zHP9he1HBa+f4h01oPVQzvaR5beVMnD20v5T3rM4kICeRwZQt5SZEEBXq/efailChO1rXT0+dBVTla3critGhWZ468sK+8qZOn9lTwnvxMvv7OZfzvB9fx/ZtWzdkFdr4y1nkY3zqTB1bV1xjnKFdVvQe4Z4TyQmD1mTy/P6xy/1j3lzd53bw2ZqbyeJR/+8tBKpu7+OIj+/jb3ZsGu2/2lTUTHxFMY4fTNXXBBM6q/vuhanr7lbsuXUh7Tx/P7K+a8P/LotQo+jxKcX070WHBNHf2sjQt+rS9nt6+xFkf8dzBKr7y2H5E4KMX5hEQIFy98szHLc5GEzoPw4xtQXIk4cGB7JuiRUXG+NNju8s5WNHCB8/N5mRdOz/afGzwtr1lTVywIIl5sWEcqJjY3/ve0iZSokPJTojg8mWpAKdt8OeNRSlOJ8fxmjaOuntBLU6NJio0iIXJUexzWxhljR18+oFdpMeF8dfPXETuGUzDNV7MkjLeCwoMYEVGjCUMMyf8/MUCVmXG8u3rV+BR+OWrhVy9ch7ZCRGUNXbyofNy6O7r5+AEu6T2ljWxKjMOEeHSpamszozlYrc14K0FyVGIONt6RIY4b2NL3O09VmfF8dLRGlSV/33pBAEi/PLD+cyLDR/rIY0XrIUxxc6ZF8OxqlY7UMnMak0dPRTWtXPNynkEBAhfuWYpKdFhfPGRvYOzkFZlxrIsPZbC2jY6e8Ze/zCgpauXE7XtrMly1lLEhgfz5F0XsT4nfkLxhYcEkhUfwfGaNo5UtZISHUp8ZAjgJIy6th5eOFzDn3eUcXN+piWLKTJuC0NEQnFWfOcOra+q/+67sGav3MRIWrv7qG/vmZMHqJizw4Fyp9WwIt15Y48JC+Z7717BR3+7g3/7yyEiQgJZmRFLa1cfHoXDVS2syx7/Tf+A2/oeGO87E4tTo9h2soHw4ECWpJ2ah7PejePjv99BUIDwyYsXnPFzGYc3XVJPAs3ATmDqNqGfo/KSnT7Sorp2Sxhm1hoYl1ieHjNYdunSVH730Y0EBQjL5sUQHRbMigx31XV5s1cJY487trBq2GZ/k/Gxi+bzmT/tpqShgyvcsRCAZekxPHLn+RTXd5AaEzZ4yJk5c94kjExVvcrnkcwReYlOwiisayffXcFqzGxzoLyZjLjwwW6eARcvPn1xbHpsGHERwV6PY+wtbSI3MYK4iJDxK4/j/AWJvPSFt/P4rjIuOyf1tNvycxPs/88HvBnD2CIiK30eyRyRGR9OUIDY+RhmVjtY0cLKjPFbASLC8vQYrxJGv0fZXdLE6qwz744aEBUaxK3n55IeZ2MU08GbhHERztbkR0Vkn3tGxT5fBzZbBQUGkJUQQVG9JQwzO7V29XKyrp0VGTHjVwaWp8dytKqV3n7PmPWe2V9JTWs3Vy5Pm4owjR940yV1tc+jmGPykiIprLWEYWangdbCci9aGOCMc/T0eyioaeOceSMnGY9HuecfBSxMieIqSxiz1rgtDFUtBuKA69xLnFtmRpGbGElxfYdNrTWz0gF3n6iBGVLjGRgYH7jfSP68s5Sj1a3cdcnCwS3BzewzbsJwj1Z9AEhxL/8nIp/xdWCzWV5SBJ29/VS32KQyM/vsK2smPTaM5GjvZvnlJUURHhw44jhGT5+HTz+wiy89up/VmbFcu8q25JjNvOmS+hhwrqq2A4jI93F2rx33XO+z1cD2Ayfr2gfPIjZmtthbNrGB6cAA4Zx50ewsbqSzp5/wkFPbhT93sIqn91fymUsX8plLF01og0Ez83jz2xNg6DLOfsbZVPBsN3DcY2Gd1yfaGjMjNLb3UFzfMeGZTJedk8r+8mbO/d7feb2gbrD8id3lpMWE8dl3LCYkyJLFbOfNb/A3wFYR+ZaIfAt4E+ecCzOK9NhwUqJD2Xyo2t+hGDMhA9uCr57gSuxPvX0BD3/ifMKCA/ntliIAGtp7ePlYLdevSSfQxi3mBG8GvX8I3A40AI3A7ar6Y18HNpsFBAgfPDeHl47WUlhrrQwzs7R09Y54wBA44xcip87N9paIsDEvgcvOSeWNE/X09nt4el8FfR7lhrUZUxG2mQHGOqI1xv2aABQB/wf8ASh2y8wY3n9uFsGBwu/fsAllZmZo6+7jmp+8yqpvPc87fvgyfUPWTXzvmcPcdv823jhRz8LkKKJCJ7eR9cWLk2jr7mNncSMPbi9lSWr0qFNtzewzVgvjj+7XncCOIZeB780YUqLDuHZVOo/sLPN6J09jfOlETRuHKltYlx1HfXsPB9xZTR6P8ucdpbx8rJY3CuvPaCX2BQuTCAwQvvP0IQ5WtPDxTXlTFb6ZAUZNGKp6rfs1T1XnD7nkqer86Qtx9rpm5Tzauvs4OMEDZozxhfp2Z5r3J9++EIA3C+sBOFrdSmNHL+9w92PaeAZ7MMWEBbM2K44D5S3MT47kRuuOmlO8WYfxgjdl5q0G9uLZP8aCJmOmS32bM26xNC2aBcmRgwnjjRPO13+7fjmvfvESblqfeUbP8zZ3g8LPvWOxTaOdY0btqBSRMCACSBKReE5NpY0B0qchtlkvNSaUpKhQSxhmRqh3B7oTo0I4b34iT+6poK/fwxuF9WQnRJAxRRv43XZ+LvNiw3innZs954yV/j+BM16x1P06cHkS+JnvQ5v9RISVGTEcLJ/YEZbGTIWG9p7TNgSsb+smLDiAiJAgzp2fSFt3H/vLm9laWM/58xOn7HljI4J5T36WbQEyB401hvETVc0D/nXI2EWeqq5W1XumMcZZbWVGLMdrWm3g20yrxvYeLv7Bi9z3SuFgWX17D4mRznYf5+U54xRffGQfLV19nL9g6hKGmbvGnTunqv8jIiuAZUDYkPLf+zKwuWJFRiwehUOVLRM+t9iYyfrjthJau/rYU9o0WFbf1kNSlHNwUUpMGO/fmDV4PsXwg5GMGYk3Z3p/E3g7TsJ4Bme789cASxheGFgAdaC82RKGmRY9fR5+5662Pl7dOlhe395N8pBjg//fu1dNd2hmlvNmCsPNwGVAlareDqwG7LBqL6XFhJEUFWID32baPL2/gprWbtZkxVHc0EFXr9Md2tDWQ6KdM2/OgDcJo1NVPUCfu/q7BrB1GF4SERanRnPCtggx0+SZ/VVkxIXzsYvyUIWCmjZUlbr2HhKjzvwsbXP28iZh7BCROOCXOLOkdgHbfBrVHJOTGEFJfYe/wzBnAY9H2V7UwIULE1maFg3A8ZpW2rr76OnzkBhpCcNMnjebD35KVZtU9V7gcuA2t2tqTCKSJSIvishhETnoHsQ0vM7bRaRZRPa4l28Mue0q9xzxAhH58kR/sJkkOyGS+vYe2rr7/B2KmeMKatto6uhlQ24CuUmRBAcKx6rbBhftDcySMmYyxlq4t26s21R11ziP3Qd8XlV3iUg0sFNENqvqoWH1Xh3YhmTI4wfirPW4HCgDtovIUyPcd1bISYwAoLi+neVeHntpzER4PIoIbD3ZAMC5eYkEBwaQlxTJ8erW0xbtGTNZY82S+m/3axiQD+zFWe29CtgKXDTWA6tqJVDpXm8VkcNABuDNm/5GoEBVCwFE5EHgei/vO+NkJzgJo6S+wxKGmXLt3X184JdvMi82nMBAIS0mjKwEZ9X2otRo9pc1U9/m7COVZIPe5gyMtXDvElW9BCgG1qlqvqquB9YCBRN5EhHJde+3dYSbzxeRvSLyNxFZ7pZlAKVD6pS5ZbPSYAujwcYxzNRSVf71z3vZW9bMswereGZ/JRvzEhBxVlkvTommtLGDssZOABJsDMOcAW8GvZeq6v6Bb1T1ALDG2ycQkSjgUeCzqjp8j4xdQI6qrsY5I/yJgbuN8FA6yuPfISI7RGRHbW2tt2FNq+iwYBIiQyi2gW8zxZ7aW8HfDlTx1WuWcs3KNFRhQ96p3WbX58SjCg/vcD5/WcIwZ8KbU1IOi8ivcA5QUuBDwGFvHlxEgnGSxQOq+tjw24cmEFV9RkR+LiJJOC2KrCFVM4GKkZ5DVe8D7gPIz88fManMBNkJEZQ0tPs7DDPHbCmoJy4imH/aNJ/W7j5yEyO5btWpTf8uWJAAKM5sAAAgAElEQVRIXlIkR6paiQ4NIiw40I/RmtnOmxbG7cBB4G7gszjjCN7MkhKcs78Pu8e8jlQnza2HiGx046kHtgOLRCRPREKAW4CnvIh1xspJjLAWhplye8uaWJUZh4gQExbMF69aSlzEqVZEQIDw4fNzAEiwAW9zhrzZS6oL+JF7mYgLgVuB/SKyxy37KpDtPu69OKvIPykifUAncIuqKs4iwbuA54BA4H5VPTjB559RshMi+MveCnr6PIQE2RkB5sx19vRzvKZt8OCj0dy0PpMfPHfUuqPMGRtrWu3DqvpeEdnPCOMHqjrmRjSq+hojj0UMrXMPMOLOt6r6DM7eVXNCdkIEHoXypk7ykiL9HY6ZAw5WNNPvUVZljj3zLiYsmO/duJKwYPugYs7MWC2MgYV2145Rx3gpJ9FJEkV17ZYwzJTYV+bsT+bNGdw32FGpZgqMmjDcdRSoavH0hTN3nTMvmgCB3aVNXLI0xd/hmDlgX1kTqTGhpMaEjV/ZmCkwVpdUKyNPZRVAVTXGZ1HNQdFhwZwzL4YdRQ3+DsXMEu3dfUSGjt4JsK+smVWZ47cujJkqYy3ci1bVmBEu0ZYsJmdDbgK7S5pOOzbTmJE0dfSQ/52/89MXjo94e3NnL4V17aweZ/zCmKnk9SiYiKSISPbAxZdBzVUbchPo7O3nYIWd8W3Gtq+smc7efn7892PsLG58y+27S5yytdl2KJeZPuMmDBF5l4gcB04CLwNFwN98HNectCHX+efeftK6pczYBj5UpMaE8bmH9tDTd3qrdFdJEwHi3YC3MVPFmxbGt4HzgGOqmodz+t7rPo1qjkqJCSMnMYJtNo5hxnGwopmMuHC+c8MKSho6+NuBytNu313SyJK0GKLGGOMwZqp5kzB6VbUeCBCRAFV9kQnsJWVOd25eAq8X1HHIuqXMGA5VtLA8PYZLlqSQlxTJb90zugH6PcrukibWZVvrwkwvbxJGk7uB4CvAAyLyE5yzLswk3P2OxcSGB/Ph+7fZKXxm0HefPsSXH93H1sJ62rv7OOmenTKwtcfukib2ljYBp07QW59j4xdmenmTMK7H2bbjc8CzwAngOl8GNZdlxIXzh49tpKOnj5+/NKFd4s0c1dXbz69fO8mD20t5331v8qPNx1CF5enOZMSb12cSFRrEd585TFdvP7uKncSxzga8zTQbNWGIyD0icoGqtqtqv6r2qervVPWnbheVmaSFKdFcvDiZl47W4mydZc5mhbXteBT+6z2rWZERw69eOwnA8gwnYUSHBfPtG5az7WQDt92/jZ+9WEBSVMjgOSvGTJexWhjHgf8WkSIR+b6I2LjFFHr7kmSqWro4Wt1KUV07xfW29fnZ6nhNKwCrMmP5j3evIjBASIgMIW3ICu4b12byjWuXsfVkAykxofz0/WsHD0kyZrqMtTXIT4CfiEgOzvbivxGRMOBPwIOqemyaYpyT3rY4GYC/7q3k4R2lZCdE8MgnL/BzVMYfjle3ERgg5CZGEhIUwDeuXUZnb/9bEsJHL8rjxrUZxNuus8ZPvNnevBj4PvB9EVkL3A98E2fbcTNJ82LDWZoWzc9eKkDV2apaVe1T41noeE0ruYkRg9ve33ZB7qh1LVkYf/Jm4V6wiFwnIg/gLNg7Btzk88jOAhcvSUYVkqJCae3uo6a1298hGT84XtPGopRof4dhzLjGGvS+XETuxzku9Q6csykWqOr7VPWJ0e5nvPe+/CyuX5POt69fDkBBTZufIzLTrbuvn+L6DhanRvk7FGPGNVYL46vAG8A5qnqdqj6gqjYyO4XmJ0fxk1vWss6dT28J4+xzsq6dfo+yMNVaGGbmG2vQ+5LpDORslhIdSnRYkCWMOay+rZv/3nyMl4/W8tAnziMz3pkSe7za+Z0vSrEWhpn57MzGGUBEWJgSNTi90swtdW3dXPWTV3l4eynVLV387MUTg7ftKmkkMEDsFEYzK1jCmCEWJkdRUGM9fnONqvL1Jw7Q3NHLY5+6gFs2ZvHIzlLKmzqpbO7kj1tLuHbVPMKCbdKhmflsq8sZYmFKFH/eWUZzRy+xEcH+DsdMkWcPVPG3A1V88aolrMqMIzEqlIe2l/KlR/YRHhKIKvzrFUv8HaYxXrEWxgyx0O3DLqi1bqm55NmDVaTGhHLHpvmAs5fYl65ays7iRjYfqua2C3LISrAtPszsYC2MGWJgHv6x6jbW5ySMWOflY7Xk58SPec6zmVlKGjpYkBxFUOCpz2Yf3zSf927I4vXjdVyyNMWP0RkzMdbCmCEy48OJCAnkaJXTwthZ3EhVc9fg7TWtXdx2/zb+vKPUXyGaSSip7xhxk8CYsGCuXmljF2Z2sYQxQwQECItSozla1Upvv4dbf72Vn7xwfPD26mZnFXhJQ6e/QjQT1NbdR317j3U5mTnDEsYMsjQ1mmPVrRypbKWjp5+iulOzpmrbnNZGeZMdujRblDY4v6tsSxhmjrCEMYMsToumvr2HzYeqAKf/e0Ctu89UeZO1MPyhqK6dvn7PhO5TYgnDzDE+SxgikiUiL4rIYRE5KCJ3j1F3g4j0i8jNQ8r6RWSPe3nKV3HOJEvTnIHvh3eUAVDZ3Emv+yY1mDAaTyWM0oYOfv9G0WAd4xvVLV1c/qOX+eO2kgndb+AI3pwEW5Rn5gZfTrfpAz6vqrtEJBrYKSKbVfXQ0EoiEoizffpzw+7fqapn1aFNS9yEUdXSRVCA0OdRKpo6yUmMpK6tB4DGjl46evr4zetF/GjzMfo8SkZcOJedk+rP0Oe0LSfq6O1Xtp5s4MPn53p9v5KGDmLCgmxdjZkzfNbCUNVKVd3lXm8FDgMZI1T9DPAoUOOrWGaLpKhQEt3zDgYOWBro1qgdsvV5UV0HP33hOPm5zqaFx6ptDypf2lLgnEi8p6RpsGxHUQNffGQvNS1do92NkoYOsu0YVTOHTMsYhojkAmuBrcPKM4AbgXtHuFuYiOwQkTdF5AafBzlDDLQyrl+TDpyeMELdA3ZePFpDd5+HD5ybQ1pMGMerbbGfr6gqW07UEyDO+FFtazf/9dxRbr73DR7eUca///UQ7d19fOWxfewuaTztvqUNHTZ+YeYUn68AE5EonBbEZ1W1ZdjNPwa+pKr9I5w0l62qFSIyH/iHiOxX1RPDK4nIHTjndZCdnT31P8A0W54ew47iRt5xTiohgQGnEkZbNyszYtlR3MjfDlQCsCYzjkWpURy3XW59prShk/KmTq5bnc5f9lbwzP5Kfv5SAdetTic9LoxfvFxIQU0bR6paERHWZjutvn6PUtrYwRXL0/z8ExgzdXzawhCRYJxk8YCqPjZClXzgQREpAm4Gfj7QmlDVCvdrIfASTgvlLVT1PlXNV9X85OTkqf8hptmnL1nIo3deQGRoEJnx4YNTM+tau1meHkNQgHCgvIX4iGCyEsJZlBJNQU0bHo/6OfK56Y3COgDu2DSfwADhv54/igJfumoJn3vHYrISwjlS1UpSVCh7S091WVW1dNHbr9bCMHOKz1oY4jQZfg0cVtUfjlRHVfOG1P8t8FdVfUJE4oEOVe0WkSTgQuA/fRXrTBIXEUJchDOOkZUQQUlDB509/bR295ESE8a8uDBKGzpZnRWHiLA4NYrO3n7KmzptgZgPvHGinqSoUFZkxLA0LZqDFS1cujRl8DyL33xkI+VNnWw7Wc8vXi6kq7efsOBAjrkr9ucn2wwpM3f4soVxIXArcOmQ6bHXiMidInLnOPc9B9ghInuBF4H/GD676myQnRBBaUMndW3OgHdydCjpseEArM6MA2CRe7TnMRvH8ImDFS2syYpFRFiT5bzmHzrvVNfnwpQoLl6czKrMOPo8ysEKp9d1T2kTAQIrM2L9ErcxvuCzFoaqvga8ZWBijPofGXJ9C7DSB2HNKlkJ4TR39lJQ64xRJEeHkhEfDicZfPNaOGTTQptaO7W6+/oprGvniuXO6/q+DVkEiHDx4rduGDjw+9hb2sT6nHj2lTWxMCXKNoo0c4r9Nc9guYlOd8bLR2sBSI4KJTshAhFYlel8co0ND3ZmStlpfVOmrq2bqNAgTtS20e9RlqbFALAqM45VbstuuNSYMNJiwthX1oSqsresmctsJ1ozx1jCmMEuWpREVGgQD213dqhNjg7ltvNzyc9JIDEqdLDeotQo65KaIqrK9fe8zsVLklnvzng6Z160V/ddnRXL3rJmyho7aWjvYVXWyMnFmNnK9pKawSJCgrhudTqdvf2IQEJkCPGRIVy0KOm0eisyYjlS2UpnT7+fIp07Kpu7KG/q5LkDVRyqbCEkKGCwpTee1VlxnKxrH9yCfs0orRFjZitLGDPc+zZkAZAQEUJw4Mi/ro25CfR5lD1DpnWaydlf3gxAfXsPT+wuZ1HK6YcfjeXm9ZkkRYXy038UEBIYMLgI05i5whLGDLc6M5aladGkxoSNWmddTjwisL2oYRojm5sOljcTIBAUINS390zoTT8lOoyfvn8NAQLnpMcQEmT/XmZusTGMGU5E+PkH19HVO/qOtLHhwSxJjbaEMQX2lzezKCWa5OhQXiuo4xx3wNtbFyxI4p4PrCPeXUtjzFxiCWMWmJ8cNW6dDbkJPLarjL5+j9ddKAb2lTXR51HWuQPcBypaeNuiZFZlxvJaQd2kupWuWTlvqsM0Zkawd5Y5YkNeAu09/ewta6bftgnx2uce2sOn/m8X/R6luqWL2tZuVmTE8J78TL5+7TIuWJDo7xCNmTGshTFHbHC3Or/pf7eQGBnCK1+8xBaNjaOorp0Ttc4xuG+cqKe7z5lltjIjloiQID52Ud5YdzfmrGMtjDliXmw493xgLe/fmE19e8/gFhVmdC8ccY5gCQsO4NFdZTyzv8oZsJ43sXELY84W9hF0Drl2VTobchP407YSDlY0szEvwd8hzRi9/Z63TEv+x5FqFqVEkZ+bwEPbS/AofOLi+dYyM2YU9p8xx6REh5IUFWItjCH2ljbxnl+8wfykSK5cnsaVy9NIjApha2EDH9uUxxXL0vjTthIuXZrCF69c6u9wjZmxLGHMMSLCsvRYDrgL0MbS2dPPlx/bx+cvXzJnjxLt9yhfe2I/MWHBxIQF89N/HOcnLxwfvP2ypamsz4nnkTvPZ3l6LIEBXu+XacxZxxLGHLQiPYb7Curo7usnNChwsPz1gjoWpzprDAB2Fjfy5J4KlqbF8Mm3L/BXuD71wNZiDpS3cM8H1nLtqnRqW7t58UgNLV29JESGDE4WyM+17jtjxmMJYw5anh5Ln0c5VtXGSndX2/buPj58/zY+emEuX3vnMgAOVTqtEG9aI7PVg9tKWZsdxzvdtRHJ0aG8191uxRgzMTZLag5anu7M8jlYcSoRHKlqpd+jFAw5/3tgnONAxdxNGNUtXZwzL4YRzow3xkyQJYw5KDshgqjQoNMGvg9VOtcL69pPlbm3F9d30NzZO71BToOePg/17T2kRo++D5cxxnuWMOaggABhVWYsO4obB8sGkkNpQwfdff109fZzorZt8KS4g3OwlVHrHm2bEhM6Tk1jjDcsYcxRFyxI5HBlC/Xum+ahyhZEwKNQUt/BkapWPArvzXf68w+Wz71puNUtXQCkWsIwZkpYwpijLljoHLL0ZmEDff0ejlS2sCHHmQlUWNc+2OLYtCiJ9NiwwXMg5pIaN2GkWJeUMVPCEsYctSojlqjQIF4/UUdRfTvdfR7eucqZKVRY287Bimaiw4LIjA9neYZ36zZG09rVy5N7ynn5WC0tXTNnLKSm1WldjXWWiDHGe5Yw5qigwADOzUtgS0Hd4OD3xrwEkqNDKahp440T9axIj0VEWJsdR2FdO7XuG+xE/XFrCXc/uIfb7t/G1x4/MJU/xhmpbukiMEBIjLSzKYyZCpYw5rALFiZRVN/B/7hHhi5IjmJ+UiRP76+gsK6d95+bDcCmhckAvHq8dlLPc7ymjaSoEN62OJl9ZTPnmNjqlm5SokMJsNXbxkwJSxhz2GVLUwgPDiRA4AtXLiEkKID5yZF09XrITojgmhVpgLNuIzEyhFeOTS5hnKxrZ0FyFBty4imu76C9u28qf4xJq27pIsW6o4yZMrbSew7LTYrk0L9fedqitflJzul9n7h4/uDJfAEBwkWLknj1eB0ej074E/nJunauXJ7KUndb8KPVrYMn2PlTTUs3OXN0jyxj/MFaGHPc8BXO71w1jzveNp+b12eeVv62RcnUt/cMLvDzVlNHDw3tPcxPimKpe5zpkcrWMwt6ilS3dtkaDGOmkCWMs0x6XDhfveac0zYlBNi02JmG+8oExzEGVo7nJUWSGR9OdGgQR6paKKprZ8uJuqkJehK6evtp6ui1Vd7GTCGfJQwRyRKRF0XksIgcFJG7x6i7QUT6ReTmIWW3ichx93Kbr+I0jpToMBYkR7KreGKD1ifdI07nJ0ciIiydF82hihY+/cddfOQ322nq6PFFuOOqtSm1xkw5X7Yw+oDPq+o5wHnAp0Vk2fBKIhIIfB94bkhZAvBN4FxgI/BNEfF/p/gctyozjv3lE0sYhXVtBAYIWQnOWMHStBh2FDdysKKFnj4Pj+4qP61+T5+H/37+6OAK9JEU1DgbJQI8e6ByUtuW1LS6i/asS8qYKeOzhKGqlaq6y73eChwGMkao+hngUaBmSNmVwGZVbVDVRmAzcJWvYjWOlRmxVLd0D26p4Y2Tde1kJ0QMHn+6dJ4zjpGXFMnqrDge2FqMqg7Wf72gjv/5RwF/2Vsx4uMdrWrl8h+9wh/eKKK5s5fP/Gk3n39472mPAfDikRqu+vErpy04LKxtG5waXN1iLQxjptq0jGGISC6wFtg6rDwDuBG4d9hdMoDSId+XMXKyMVNolXt2xv4y7z/RF9a2Mz8pcvD7tVlOQ/BfLl/Mh87NprC2nTcLGwZvHxjX2DfKczyysxRVeHRXOZsPVdPbrxypauUfR2pOq/fwjlKOVLXyvl+8wZuF9QB886mD3Hb/Nl49XstvXj9JREjgYMvHGHPmfJ4wRCQKpwXxWVUdPgXnx8CXVLV/+N1GeCgdoQwRuUNEdojIjtraya0jMI5l6TEECOwbZZuQzp5++vo9g997PEpRfTt5QxLGsvQY3vzKZVy3Op1rV6WTGBnCt/96iO4+51e85YTz5j7Sc/T1e3h8dwVhwQHsL2/m3pdPkBEXTkZcOD9/6cRgK6Pfo2w5Uc9lS1NIjg7lW08dpKmjhy0n6lHgI7/ZzvaiRv7fu1cSFWozx42ZKj5NGCISjJMsHlDVx0aokg88KCJFwM3Az0XkBpwWxdBj0TKBEfswVPU+Vc1X1fzk5OQpjf9sExESxKKUaPaPsFq7o6ePy3/0Mpf+98v8/VA14MyQ6ur1sDg1+rS6abFON1B4SCD/cdMqDlW28MPnj9HU4UzbjQkL4kRtG23uAr8jVS1c9eNX+MZTB6lr6+Zr71xGgEBBTRvXrnamAe8sbmTTf77Ir14tZH95M82dvbxrTTqfuHgBR6pa+c/njtLvUX5w82oiggO59bwcrl9jjVJjppLPPn6JswDg18BhVf3hSHVUNW9I/d8Cf1XVJ9xB7+8NGei+AviKr2I1p6zMjOWlozWo6mlrOO596QRljZ3kJEbw8d/v4PFPXcBx9/S+dTlxoz7e5ctS+eC52fzilUJq27pRhQ+dl8PPXzrBwfJmzp2fyOaD1RypauVIVSvxEcG8Lz+LzYeqeeVYLdeuTGdZegyhQQE8tquc7zx9mMuXpQJw4cIkwoID+c5fD/HHrSVkxIVz07oMrlmZRnhw4KgxGWMmx5ctjAuBW4FLRWSPe7lGRO4UkTvHuqOqNgDfBra7l393y4yPrcqMpa6th/KmzsGy0oYOfvFKIdevSecvn7mIwADh74er2V3SRExY0ODq8dF8/dpl5OfE89iucsKDA7n1/ByAwS3V95c3k5cUyT0fWMtPbllLSFAAd1+2kH/alMeKjBgCA4RbNmZz/+0bSI0JZfOhapbNiyEpKpSo0CDe5bYkrlyehogQERJkR7Ia4wM+a2Go6muMPBYxWv2PDPv+fuD+KQ7LjOO8+YkAvHi0llvPc97Yf/lqIQp8+eqlxIQFsy47jleO1dHb72FNdvy4W4mEBQfyyw/nc9O9W1iUEsW82HDSY8MGB74PlDeTn5vAtavSB++zPieB9e75HQOiQoP46jXncPeDe9i0KGmw/LYLcnjuYBXvXmddUMb4ko0ImtMsSnF2tH32QCW3npdDd18/T+2t4MrlacyLDQdg06JkfvT3Y4Dzqd4b8ZEhPPPPmwh0k8vKzFj2ljVR39ZNRXMXKzNivXqcd61Op6u3n0uWpAyWLU2LYdfXL5/Ij2mMmQTbGsScRkS4akUabxY20Njew4tHamjq6OWmIZ/e37Y4GVVQhXU53q+nDAsOHFyvceHCJIrrO/jTthIAVniZMESE923Itl1ojfEDSxjmLa5eMY9+j7L5cDWP7ionOTqUixae6gJamRFLXEQwAGuyRh/wHssNazMIDw7kf/5RAMDyjJgzD9wY41OWMMxbrMiIITM+nP/v8QNsPlTNjWszBrdCBwgMEK5YlsqarDhiw4Mn9RwxYcHcsDaD7j4PeUmRxIRN7nGMMdPHxjDMW4gI37xuOf84UkNydCgfdmc1DfXdG1cO7vc0Wbeel8OftpWwPN1aF8bMBpYwzIguX5Y6uN5hJMGBAZzpUodl6TF8+eqlbMxLGL+yMcbvLGEYv7rz4gX+DsEY4yUbwzDGGOMVSxjGGGO8YgnDGGOMVyxhGGOM8YolDGOMMV6xhGGMMcYrljCMMcZ4xRKGMcYYr8jAOclzgYjUAsX+jmMUSUCdv4MYg8V35mZ6jBbfmZvpMU4mvhxV9ep86zmVMGYyEdmhqvn+jmM0Ft+Zm+kxWnxnbqbH6Ov4rEvKGGOMVyxhGGOM8YoljOlzn78DGIfFd+ZmeowW35mb6TH6ND4bwzDGGOMVa2EYY4zxiiWMKSYiWSLyoogcFpGDInK3W/4tESkXkT3u5Ro/x1kkIvvdWHa4ZQkisllEjrtf4/0U25Ihr9MeEWkRkc/68zUUkftFpEZEDgwpG/H1EsdPRaRARPaJyDo/xvgDETnixvG4iMS55bki0jnktbzXT/GN+jsVka+4r+FREbnST/E9NCS2IhHZ45b74/Ub7b1l+v4OVdUuU3gB5gHr3OvRwDFgGfAt4F/9Hd+QOIuApGFl/wl82b3+ZeD7MyDOQKAKyPHnawi8DVgHHBjv9QKuAf4GCHAesNWPMV4BBLnXvz8kxtyh9fwY34i/U/d/Zi8QCuQBJ4DA6Y5v2O3/DXzDj6/faO8t0/Z3aC2MKaaqlaq6y73eChwGMvwbldeuB37nXv8dcIMfYxlwGXBCVf26IFNVXwEahhWP9npdD/xeHW8CcSIyzx8xqurzqtrnfvsmkOnrOEYzyms4muuBB1W1W1VPAgXARp8Fx9jxiYgA7wX+5MsYxjLGe8u0/R1awvAhEckF1gJb3aK73Kbh/f7q7hlCgedFZKeI3OGWpapqJTh/nECK36I75RZO/yedSa/haK9XBlA6pF4ZM+NDw0dxPnEOyBOR3SLysohs8ldQjPw7nWmv4SagWlWPDynz2+s37L1l2v4OLWH4iIhEAY8Cn1XVFuB/gQXAGqASp3nrTxeq6jrgauDTIvI2P8fzFiISArwL+LNbNNNew9HICGV+nY4oIl8D+oAH3KJKIFtV1wL/AvxRRGL8ENpov9OZ9hq+n9M/uPjt9RvhvWXUqiOUndFraAnDB0QkGOcX+oCqPgagqtWq2q+qHuCX+Lh5PR5VrXC/1gCPu/FUDzRZ3a81/osQcJLZLlWthpn3GjL661UGZA2plwlUTHNsg0TkNuBa4IPqdm67XT317vWdOGMEi6c7tjF+pzPmNRSRIODdwEMDZf56/UZ6b2Ea/w4tYUwxt6/z18BhVf3hkPKhfYc3AgeG33e6iEikiEQPXMcZGD0APAXc5la7DXjSPxEOOu1T3Ux6DV2jvV5PAR92Z6mcBzQPdBlMNxG5CvgS8C5V7RhSniwige71+cAioNAP8Y32O30KuEVEQkUkz41v23TH53oHcERVywYK/PH6jfbewnT+HU7nKP/ZcAEuwmn27QP2uJdrgD8A+93yp4B5foxxPs4MlL3AQeBrbnki8AJw3P2a4McYI4B6IHZImd9eQ5zEVQn04nxy+9horxdOV8DPcD517gfy/RhjAU4/9sDf4r1u3Zvc3/1eYBdwnZ/iG/V3CnzNfQ2PAlf7Iz63/LfAncPq+uP1G+29Zdr+Dm2ltzHGGK9Yl5QxxhivWMIwxhjjFUsYxhhjvGIJwxhjjFcsYRhjjPGKJQxjxiEi/XL67rlfnsLHzh26O6oxM1mQvwMwZhboVNU1/g7CGH+zFoYxk+Sej/B9EdnmXha65Tki8oK7od4LIpLtlqeKcybFXvdygftQgSLyS/eMg+dFJNyt/88icsh9nAf99GMaM8gShjHjCx/WJfW+Ibe1qOpG4B7gx27ZPTjbSq/C2ezvp275T4GXVXU1zrkLB93yRcDPVHU50ISzihicsw3Wuo9zp69+OGO8ZSu9jRmHiLSpatQI5UXApapa6G4KV6WqiSJSh7PFRa9bXqmqSSJSC2SqaveQx8gFNqvqIvf7LwHBqvodEXkWaAOeAJ5Q1TYf/6jGjMlaGMacGR3l+mh1RtI95Ho/p8YW34mzF9B6YKe7a6oxfmMJw5gz874hX99wr2/BOfgJ4IPAa+71F4BPAohI4FjnJ4hIAJClqi8CXwTigLe0coyZTvaJxZjxhYvIniHfP6uqA1NrQ0VkK86Hr/e7Zf8M3C8iXwBqgdvd8ruB+0TkYzgtiU/i7I46kkDg/0QkFmfX0R+patOU/UTGTIKNYRgzSe4YRr6q1vk7FmOmg3VJGWOM8Yq1MIwxxnjFWhjGGGO8YgnDGGOMVx0pvEAAAAAhSURBVCxhGGOM8YolDGOMMV6xhGGMMcYrljCMMcZ45f8HAZ1/ddLyttwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def smooth_curve(points, factor=0.9):\n",
    "    if len(points) == 0:\n",
    "        return []\n",
    "    smoothed_points = [points[0]]\n",
    "    for point in points[1:]:\n",
    "        pre = smoothed_points[-1]\n",
    "        smoothed_points.append(pre * factor + point * (1 - factor))\n",
    "    return smoothed_points\n",
    "\n",
    "smooth_mae_history = smooth_curve(average_mae_history[10:])\n",
    "plt.plot(range(10, len(smooth_mae_history) + 10), smooth_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到模型在大约 60 个 epoch 之后就开始出现过拟合，可以重新训练一个模型，将 epoch 设置为 60。\n",
    "\n",
    "### 训练最终的模型\n",
    "\n",
    "训练模型，并在测试集上评估性能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102/102 [==============================] - 0s 2ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "2.489868173412248"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = build_model()\n",
    "model.fit(train_data, train_targets,\n",
    "          epochs=60, batch_size=16, verbose=0)\n",
    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)\n",
    "test_mae_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最终在测试集上得到了 2.49 的 MAE，MAE 为 2.5 表示平均误差为 2.5。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "22.395049504950492"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_targets.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里目标值的均值为 22.4 而平均误差为 2.5。这效果是好是坏，就要根据实际项目而论了。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
